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1 . INTRODUCTION 


1 . 1 Teleoperators  and  Supervisory  Control 

This  study  is  neither  an  experimental  research  report  nor  a survey  in  the 
usual  sense.  It  is,  rather,  a broad  examination  of  future  undersea  tasks  the 
U.  S.  Navy  faces  and  the  potential  for  their  accomplishment  by  "teleoperators". 

We  define  teleoperators  to  be  general  purpose  submersible  work  vehicles  con- 
trolled remotely  by  human  operators  and  with  video  and/or  other  sensors,  power 
and  propulsive  actuators  for  mobility,  with  mechanical  hands  and  arms  for 
manipulation  and  possibly  a computer  for  a limited  degree  of  control  autonomy. 

A manned  submersible  is  not  a teleoperator  vehicle,  but  the  attached  manipula- 
tors are  certainly  teleoperators,  requiring  control  through  a viewing  port  or 
through  closed-circuit  video.  Sometimes  the  term  "teleoperator"  is  restricted 
to  telemanipulator,  excluding  the  system  for  remotely  positioning  and  orienting 
a sensor,  but  for  the  sake  of  generality  we  include  this  important  function. 

This  study  focuses  on  those  aspects  of  undersea  teleoperation  which  con- 
cern the  human  operator  and  the  man-machine  interface,  and  within  this  still 
relatively  broad  domain,  it  concentrates  on  the  prospects  for  utilization  of 
"supervisory  control".  Supervisory  control  is  a hierarchical  control  scheme 
whereby  a system  (which  could  be  a teleoperator,  but  could  also  be  an  aircraft, 
power  plant,  etc.)  having  sensors,  actuators  and  a computer,  and  capable  of 
autonomous  decision-making  and  control  over  short  periods  and  in  restricted 
conditions,  is  remotely  monitored  and  intermittently  operated  directly  or 
reprogrammed  by  a person. 

The  distinction  between  direct  human  control  of  a teleoperator  and  super- 
visory control  of  a teleoperator  is  made  graphically  in  Figure  1.1.  In  the 
upper  figure  the  human  directly  controls,  over  either  a wire  or  sonic  communi- 
cation link,  the  separate  propulsive  actuators  of  the  vehicle,  the  actuators  for 
the  separate  degrees  of  freedom  of  the  manipulator,  and  the  pan  and  tilt  actuators 
of  the  video  camera.  The  video  picture  is  sent  back  directly  to  the  operator. 

The  "hand  control"  can  be  a master-slave  positioning  replica  or  a rate  joystick. 
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FIGURE  1.1  DIRECT  AND  SUPERVISORY  CONTROL  OF  A TELEOPERATOR:  DEFINITIONS 

TELEOPERATOR:  A vehicle  having  sensors  and  actuators  for  mobility 
and/or  manipulation,  remotely  controlled  by  a human  operator, 
and  thus  enabling  him  to  extend  his  sensory-motor  function  to 
remote  or  hazardous  environments. 

SUPERVISORY  CONTROL:  A hierarchical  control  scheme  whereby  a 
(teleooerator  or  other)  device  having  sensors,  actuators  and 
a computer,  and  capable  of  autonomous  decision  making  and  control 
over  short  periods  and  restricted  conditions,  is  remotely  moni- 
tored and  intermittently  operated  directly  or  reprogrammed  by 
a person. 
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In  the  lower  figure  a computer  is  added  to  the  teleoperator,  and  for 
short  periods  and  limited  circumstances  this  teleoperator  can  function  autono- 
mously. 

At  the  bottom  are  generic  definitions  of  teleoperator  arid  supervisory 
control.  The  upper  drawing  portrays  the  former  without  the  latter.  The 
lower  drawing  is  the  combination. 

In  supervisory  control  the  teleoperator's  (remote)  computer  communicates 
at  high  bit-rate  with  the  teleoperator's  sensors  and  actuators.  But  because 
of  bandwidth  constraints  on  the  signa")  transmission  link,  or  because  of  tele- 
operator sensing  limitations,  communicat’on  may  be  restricted  to  low-bit-rate 
with  the  human  operator's  (local)  computer.  For  this  reason,  and  also  because  of 
the  intermittent  nature  of  human  monitoring  and  reprogramming  of  commands  on  a 
keyboard  (and  possibly  joystick  or  other  controls),  the  human  supervisor's  communi- 
cation with  the  teleoperator  tends  to  be  at  a slow  rate,  i.e.,  intermittent 
symbol  strings  or  movement  sequences  on  a master-controller  with  relatively 
many  bits  per  instruction  package.  His  communication  with  the  local  computer 
to  refresh  TV  images  or  to  edit  or  "dry  run"  his  commands  on  a model  before 
commiting  them  to  action  may  be  constrained  only  by  his  own  speed  limitations. 

The  details  of  supervisory  control  are  discussed  more  thoroughly  in  succeeding 
sections  of  this  report. 

The  physical  separation  of  local  and  remote  computer  is  not  necessary  in 
aircraft,  industrial  plants  or  other  systems  where  the  operator  is  physically 
nearby,  and  where  supervisory  control  is  used  for  reasons  other  than  physical 
remoteness  and  limited  communi cation  channel  capacity  between  human  operator 
and  the  object  of  control.  In  such  situations  supervisory  control  may  be 
advantageous,  nevertheless,  to  achieve  faster  or  more  accurate  control,  or  to 
control  simultaneously  in  more  degrees-of-freedom  than  the  operator  can  achieve 
by  direct  servo-control,  or  to  relieve  him  of  tedium.  The  latter  reasons  for 
supervisory  control  can  apply  to  undersea  vehicles  when  the  human  operator 
is  not  physically  distant  (as  with  manned  sebmersibles ) or  to  undersea  tele- 
operators when  a reliable  high-bandwidth  communication  channel  (wire  or  optical 
tether)  is  available. 
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1 2 Why  Teleoperators  Underseas?  The  Limits  of  Divers  and  Manned  Submersibles 

The  principal  reasons  for  interest  in  using  teleoperators  for  undersea 
tasks  are  dollar  costs  and  safety. 

Operations,  including  exploration,  inspection,  construction,  maintenance, 
salvage  and  rescue,  are  having  to  be  performed  at  increasing  depths.  At  such 
depths  - below,  say  300  m.  (depending  upon  the  particular  task)  the  time 
required  for  divers  - mostly  compress ion/decompress  ion  time  - becomes  excessive; 
factors  having  to  do  with  depth  per  se  including  life  support  equipment  become 
increasingly  costly;  personal  safety  is  more  and  more  difficult  to  maintain. 
These  assertions  are  borne  out  by  rather  alarming  mortality  figures  for  commer- 
cial divers  in  the  North  Sea. 

Figures  1.2  and  1.3  give  further  indication  of  the  problem.  Water 
turbidity  and  other  depth-related  factors  may  require  greater  bottom-time, 
thus  compounding  tne  decompression-time  factor.  Under  such  conditions,  a 
fixed-capability  tel eoperatorj  which  sometimes  is  seen  as  too  clumsy  by 
comparison  to  a human  diver  at  shallower  depths,  becomes  much  more  attractive 
economically. 

Happily,  there  is  progress  in  the  development  of  teleoperators,  and  they 
are  becoming  less  clumsy.  Inspection  and  manipulation  tasks  which  simply 
could  not  be  accomplished  a few  years  ago  are  now  achievable,  due  to  steady 
progress  in  the  design  of  video  systems,  mechanical  valves  and  actuators,  etc. 
For  the  immediate  future,  however,  the  primary  technological  factor  which  is 
changing  the  prospects  for  undersea  teleoperation  is  the  computer. 

Circa  1970, divers  seemed  to  have  the  edge  on  manned  work-vehicles  with 
manipulators  in  terms  of  maneuverability,  manipulation,  tactile  sensing,  and 
covertness.  Because  of  smaller  unmanned  vehicles  and  eventually  through 
unmanned  untethered  vehicles,  however,  the  diver  vespecially  the  tethered 
diver)  is  losing  his  edge.  Manipulation,  sensing  and  cognition  remain  the 
primary  advantages  fcr  the  diver,  but  the  computer  is  changing  these  also. 
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DECOMPRESSION  - The  cost  of  divert* . 


FIGURE  1.2  DECOMPRESSION  TIME  REQUIRED  BY  A HUMAN  DIVER  is  one  of  the 
major  penalties  and  costs  of  diving  at  ambient  pressures 
(Shilling,  1976).  At  first,  the  longer  the  time  spent  at 
a particular  depth  (bottom  time)  the  longer  is  the  time  spent 
in  decompression.  After  enough  bottom  time  the  blood  becomes 
saturated  with  dissolved  gases  and  the  decompression  time  is 
then  just  a function  of  depth  (approximatly  1 hour  for  every 
6 feet) . 
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FIGURE  1.3  COST  COMPARISONS  OF  VARIOUS  MEANS  FOR  UNDERWATER  WELDING  were 
estimated  by  Moore  (1975).  Note  that  the  per-hour  cost  of 
saturation  diving  is  reduced  for  longer  dives  because  the 
time  for  decompression  is  a smaller  portion  of  the  total. 

The  choice  of  diver,  manned  or  unmanned  submersible  will 
also  depend  on  their  productivities;  if  it  takes  the  remote 
work  vehicle  twice  as  long  to  accomplish  the  same  task  it 
has  no  cost  advantage  over  the  manned  submersible. 


The  comparison  between  teleoperators  and  manned  submersibles  is  more  clear- 
cut.  The  fact  is  that  television  cameras  can  now  "see"  with  less  light  than  the 
human  eye,  and  new  sonic  imaging  systems  can  see  through  densely  turbid  waters 
where  neither  human  vision  nor  video  can  function.  Spatial  resolution  of  video 
can  be  made  to  approximate  that  of  the  eye  by  focusing.  Present  advantages 
of  manned  submersibles  or  teleoperators  as  work  vehicles  (neglecting  for  the 
moment  personnel  rescue)  are:  steropsis  for  close-up  objects,  and  the  ability 
of  a human  observer  with  a wide  angle  of  view  to  keep  track  of  the  relative 
location  of  different  objects.  As  the  conminication  channel  improves,  to  the 
point  where  the  manipulator  itself  is  the  limiting  factor,  a man  at  the 
surface  can  control  manipulators  or  video  pan-tilt  controls  just  as  well  as  a 
man  in  a submersible.  The  major  differences  remaining  between  manned  submersible 
and  teleoperator  are  then  cost  and  safety,  as  with  the  diver.  The  pressure 
vessel  and  life-support  equipment  make  the  manned  submersible  much  more  costly 
than  the  same  vehicle  without  the  pressure  vessel  and  life-support  equipment 
but  with  remote  control  instead.  The  factors  of  quality  and  reliability  of 
communication  and  remote  control  then  become  the  key  factors;  these  are  discussed 
in  subsequent  sections. 

1 • 3 Why  Supervisory  Control  of  Teleoperators  Underseas?  Progress  in  Micro- 
electronics and  Computer-Control  Software. 

At  the  authors'  own  institution  just  25  years  ago,  the  Whirlwind  1 computer 
filled  a 2 1/2  story  brick  bu’lding.  Today  the  equivalent  computational 
capacity  is  available  in  a single  microelectronic  chip.  The  implications  of  this 
technological  change  for  undersea  capability  are  immense. 

Physical  size  has  decreased  to  the  point  where,  in  comparison  to  other 
physical  components  required  for  a teleoperator,  the  space  requirement  is 
negligible.  The  power  requirements  of  microelectronics,  by  compairson  to 
those  required  for  propulsion,  sonic  communication  and  other  possible  needs, 
are  also  close  to  negligible.  Computing  speeds  have  increased  also, but  not 
so  dramatically  as  size  and  power.  But  the  most  dramatic  change  of  all  has  been 
in  cost. 
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Concomitant  wi th  these  hardware  changes,  primarily  cost,  have  been 
equally  dramatic  developments  in  computer  software,  indue  ,ng  "systems  archi- 
tecture", "pattern  recognition"  and  "artificial  intelligence".  The  implications 
of  these  for  underseas  operations  are  probably  less  well  understood,  but  the 
hardware  changes  without  the  software  changes  would  not  make  the  potential 
for  teleoperation  nearly  so  great, 

It  is  probably  true  that  much  of  the  sophisticated  computer  research  now 
being  carried  out  in  academia  has  questionable  immediate  practical  value  here  so 
long  as  a hunan  operator  is  in  the  control  loop  with  fairly  satisfactory 
communication  from  and  to  the  teleoperator,  and  he  can  get  reasonable  visual 
images  at  least  intermittently  and  control  the  vehicle  and  its  arms,  if  it  has 
any,  with  fair  precision. 

But  the  point  is  that  potential  for  computer  control,  with  even  modest 
degrees  of  pattern  recognition  and  artificial  intelligence,  is  just  beginning 
to  be  understood.  This  report  is  intended  to  aid  that  process. 

In  the  next  section  we  review  some  of  the  bibl iograDhic  sources  on  the 
history  of  teleoperators,  not  only  as  applied  to  undersea  problems,  but  also 
as  applied  to  nuclear  operations,  space,  and  the  industrial  production  line. 

In  subsequent  sections  we  restate  some  assertions  about  the  problem, 
specify  the  purposes  of  cur  present  study,  and  discuss  our  framework  for 
dealing  with  these  problems  and  purposes  in  this  report. 

1 • 4 History  of  Teleoperators 

The  first  concerted  work  on  teleoperators  was  done  by  the  U,  S.  Atomic 
Energy  Commission  during  the  late  194D's  and  early  1950‘s,  and  located  primar- 
ily at  the  Argonne  National  Laboratory  near  Chicago  within  a group  headed  by 
Raymond  Goertz.  This  early  work  (Goertz,  1954)  culminated  in  the"model  3" 
tape/ cable-connected  master-slave  "through-the-wal 1 " manipulator,  still 
much  used  and  still  being  manufactured  in  modified  form  for  the  nuclear 
industry,  and  the  E-l , 2,  5,  4 series  of  force-reflecting  electrical -servo- 
controlled  master-slave  manipulator  with  controlled  articulations  in  the 
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fingers  as  well  as  the  hand.  About  the  same  time  General  Mills,  American 
Machine  and  Foundry,  and  the  Unimation  Co,,  were  starting  their  manipulator 
developments . 

The  U.  S.  Navy,  at  the  University  of  California  in  La  Jolla,  began  its 
involvement  with  teleoperators  by  developing  the  Remote  Underwater  Manipulator 
(RUM),  a servo-manipulator  attached  to  a bottom  crawling  work  platform  which 
played  out  its  own  control  cable  as  it  went  into  the  water.  By  the  end  of  the 
1950's  the  early  research  submersible,  Trieste,  had  been  equipped  with  a 
manipulator,  and  other  groups  were  planning  to  augment  their  submersibles  in 
similar  fashion. 

This  early  history  of  teleoperators  (or  "telechirics" , an  alternative  term 
which  was  also  in  fashion  at  the  time)  is  recounted  in  a number  of  sources,  in- 
cluding reports  by  USAEC  (1964),  Johnson  and  Corliss  (1967),  Corliss  and  Johnson 
(1968),  Johnson  and  Magee  (1969),  Gavrilovic  and  Wilson  (1970).  More  recent 
conferences  on  remotely  manned  systems  are  reported  in  proceedings  edited  by 
Heer  (1973,  1975),  and  others.  In  the  last  two  years,  a number  of  teleoperator 
conferences  have  been  sponsored  by  professional  societies,  including  the 
Institute  of  Electrical  and  Electronic  Engineers,  the  Society  of  Manufacturing 
Engineers  (and  its  offshoot,  the  Robot  Society  of  America),  and  various  inter- 
national societies  such  as  the  International  Federation  for  the  Theory  of 
Machines  and  Mechanisms. 

1 , 5 Summary  of  Assertions  about  the  Problem 

We  now  summarize  our  principle  axioms  or  assertions  about  the  problem: 

1.  Demands  ace  increasingly  stringent  in  terms  of  depth,  sensory  resolution, 
speed  and  accuracy  and  power  of.  response  for  accomplishment  of  undersea 
tasks.  Some  of  these  tasks  are  always  the  same  and  are  amenable  to  fixed 
automation,  but  many  are  different  each  time  they  occur  and  therefore 
cannot  be  done  by  fixed  automation. 

2.  In  terms  of  depth  and  skill  human  divers  are  reaching  their  limits,  or  when 
they  go  beyond  these  limits  they  do  so  at  significant  risk  to  life  and  cost 
in  support  equipment  and  personnel. 
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3.  Teleoperators,  i.e.,  submersibles  having  video  and  other  sensors,  actuators 
for  mobility  and  manipulation,  and  remotely  controlled  by  human  operators, 
offer  much  promise  for  extending  man's  flexible,  adaptable,  perceiving  and 
control  capabilities  into  remote  and  hazardous  environments. 

4.  Present  teleoperators  are  quite  limited  in  sensory  capability  (e.g.,  in 
turbid  water),  in  manipulation  capability  (in  speed  and  dexterity  as 
compared  to  human  hands),  and  in  dealing  with  distortion  in  man-machine 
communication  (miscrientation  of  teleoperator  to  human  body,  time  delays 
and  noise) . 

5.  Computers  are  rapidly  getting  smaller  in  size  and  power  requirement  and 
cheaper  in  cost  for  a given  computing  capability. 

6.  While  accomplishment  of  one-of-a-kind  undersea  tasks  by  intelligent  and 
completely  autonomous  robots  may  have  appeal,  we  simply  do  not  have 
available  at  this  time  such  devices  or  the  understanding  to  build  such 
devices . 

7.  Undersea  systems,  like  aerospace  systems,  demand  conservative  design  because 
unreliability  poses  severe  costs. 

8.  The  most  immediate  and  reliable  approach  would  appear  to  be  to  add  modest 
computer  aiding  and  "artificial  intelligence"  to  teleoperators,  retaining 
human  sensing,  motor,  memory  and  decision  capability,  at  least  for  higher  level 
planning,  decision-making, and  control. 

9.  Over  a longer  period  of  years,  as  computer  control  and  artificial  intelligence 
become  more  sophisticated,  certain  human  functions  in  teleoperation  may  be 
replaced,  but  greater  need  and  demand  will  be  placed  upon  other  human  func- 
tions, and  in  these  respects  the  need  for  improved  man-computer  interaction 
will  increase,  not  dimish. 

1 . 6 Purposes  of  This  Study 

We  now  summarize  our  purposes  in  this  study: 

1.  Survey  and  analyze  undersea  tasks  appropriate  to  accomplishment  by  tele- 
operators . 

2.  Analyze  constraints  in  the  undersea  environment  and  technological  constraints 
of  submersible  vehicles,  communication  and  control  systems  which  most  sig- 
nificantly mediate  teleoperator  control  - primarily  the  man-machine  aspects. 
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3.  Investigate  and  define  theories  of  operator  control  performance  applicable 
to  remotely  controlled  systems.  Develop  taxonomic  and  mathematical  model? 
of  man-machine  interactions  in  undersea  teleoperation  (inspection,  vehicle 
control,  manipulation),  particularly  those  pertaining  to  supervisory 
control  - where  man  controls  computer  on  slow  time  scale  while  computer 
controls  teleoperator  on  fast  time  scale. 

4.  Recommend  specific  laboratory  simulation  experiments  with  human  subjects 
and  software  developments  to  explore  and  demonstrate  various  supervisory 
control  modes,  and  measure  teleoperator  performance. 

5.  Perform  some  of  the  above  experiments  and  apply  some  of  the  above  models. 
(This  is  planned  for  follow-on  phases  of  the  present  contract.).-. 

1.7  Framework  for  Organization  of  the  Report 

In  Section  . 2 different  types  of  existing  undersea  vehicles  are  classified, 
using  taxonomies  or  classification  schemes  which  serve  to  illustrate  the  in- 
fluence of  several  key  control  variables,  such  as:  size  and  weight;  whether 
it  is  manned  or  unmanned;  what  tasks  it  is  designed  for;  etc.  Hypothetical 
vehicles  are  also  considered,  and  it  is  evident  that  the  unmanned,  untethered 
vehicle  is  a gap. 

Section  3 classifies  those  undersea  tasks  which  teleoperators  presently  or 
in  the  future  can  be  called  uPon  to  perform.  Various  methods  of  task  analysis 
are  illustrated  by  the  techniques  several  authors  have  used  to  analyze  under- 
sea tasks.  The  major  lessons  which  are  emerging  from  these  task  analyses  are 
reviewed . 

Section  4 deals  with  the  problem  of  control  in  a general  way;  what  it  is 
and  what  are  the  prospects  for  both  human  and  computer  contributions  to  control. 

Section  5 discusses  alternative  control  system  hardware  configurations 
from  the  viewpoint  of  their  interaction  with  the  human  operator  for:  sensing 
communication,  display,  vehicle  mobility,  manipulation,  command. 

In  section  6 we  review  and  discuss  various  computer  languages  and  decision 
aids  for  supervisory  control,  and  generalize  on  the  important  human  factors 
which  affect  their  design. 

1-11 


Section  7 discusses  problems  and  methods  for  measuring  and  evaluating  human 
operator  performance  in  teleoperator  control,  citing  various  examples  from  the 
literature  to  indicate  the  status  of  understanding  here. 

Section  8 discusses  theory  and  quantitative  models  pertaining  to  the  man- 
machine  aspects  of  teleoperator  control,  particularly  supervisory  control  of 
teleoperators. 

Finally,  section  9 presents  a list  of  important  research  needs  pertinent 
to  undersea  teleoperators,  and,  in  particular,  to  supervisory  control  of  under- 
sea teleoperators. 
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2.  TELEOPERhTED  VEHICLES 


The  purpose  of  this  section  is,  first,  to  indicate  the  range  if  undersea 
vehicles  which  presently  exist  and  how  they  vary  with  respect  to  certain  key 
variables  which  affect  their  control.  Secondly,  and  by  contrast  to  what  presently 
exists,  we  discuss  the  possibility  for  unmanned,  untethered  supervisory- 
controlled  vehicles  (teleoperators). 

2.1  Present  Undersea  Vehicles 


Table  2.1  is  a listing  of  unmanned  (remotely  manned)  undersea  vehicles 
indicating  name,  operator,  depth  and  weight.  Most  of  the  data  are  from 
Vadus  (1976).  While  we  have  added  more  recent  vehicles  known  to  us,  we  do 
not  claim  this  to  be  a comprehensive  list. 

Table  2.2  is  a listing  of  manned  vehicles,  also  from  Vadus  (1976). 

The  key  variables  indicated  which  concern  control  and  apply  to  the  sub- 
sequent sections  of  this  report  are 

1.  whether  a vehicle  is  manned  or  unmanned 

°.  its  design  depth 

3.  its  size  and  weight 

4.  whether  it  is  tethered  or  untethered 

5.  whether  it  carries  its  own  power  source  (even  if  tethered) 

6.  what  its  speed  and  operating  endurance  are  (these  are  related) 

7.  what  sensors  and  manipulators  it  carries 

Figure  2.1  illustrates  a rather  interesting  differentiation  between  manned 
and  unmanned  vehicles  in  terms  of  weight  and  depth.  As  depth  of  manned  vehicles 
increases  weight  also  increases  due  presumably  to  required  additional  strength 
of  the  pressure  vessel.  The  weight  of  unmanned  vehicles  is  not  so  sensitive 
to  depth,  and  there  is  more  variability  in  weight,  the  primary  determiner  of 
weight  being  the  function.  Note  that  unmanned  vehicles  numbered  24,  25  and  45, 
which  are  special-purpose  heavy-duty  work  vehicles,  are  exceptions  to  the  unmanned 
cluster,  and  that  "JIM"  (50),  the  atmospheric  diving  suit  , is  a sole  excep- 
tion to  the  manned  cluster. 
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TABLE  2. 1 UNMANNED  SU0MERSIBLFS,(V<idus,1976)  TABLE  2.2  MANNED  SUBHER£IBLES(V«dus ,1976) 


OEPTH 

WEIGHT 

DEPTH 

WEIGHT 

NAME 

OPERATOR 

AIL-1 

(lbs.) 

«Att£ 

OPERATOR 

1ft.) 

(lbs.) 

1 

BATFISH 

Bedford  Inst.,Caa. 

6 SO 

154 

1 NEMO 

SW  Research  Inst* 

600 

2,000 

2 

PAP 

Soclete  ECA,  Fr. 

600 

1,760 

2 SEA  EXTLORER  Sea  Line  Inc. 

600 

3.600 

3 

ROBOT 

Mitsubishi  Ind.,  Jap 

1.  COO 

3,530 

3 PC-3B 

Int'l  V.W.  Contr. 

600 

6.350 

4 

TELENANTE  I 

Inst.  Fr.  Petro. 

1,000 

2,200 

4 SEA  RANGER 

Verne  Engr.  Corp. 

600 

19.000 

5 

TELENANTE  II 

Inst.  Fr.  Petro. 

1 ,000 

2,200 

5 NEKTON  /IPHA  Gen.  Oceanographies 

i ,000 

4,500 

6 

TROV 

Canada  Center  Inland 

6 NtKTON  BETA 

Gen.  Oceanographies 

1,000 

4,700 

Waters 

1,200 

1,300 

7 NEKTON  GAMMA  Gen.  Oceanographies 

1,000 

4,700 

8 

TROV  I 

Underground  Loc. 

8 PC-8 

Intersub 

800 

11,000 

Serv. 

1,200 

2,000 

9 SEA  RAY 

Sub  RJD  Corp. 

1,000 

9.000 

9 

TROV  11 

McElhanney  Offshore 

1,200 

1,200 

10  OP  SUR 

Ocean  Systems 

l.ono 

10,400 

10 

EL  SNOOPY 

Nav.  Ocean  Sys . Ctr. 

1,500  • 

• 150 

11  AQUARIUS 

HYCO  Subsea 

1,100 

11 ,000 

11 

EL  SNOOPY 

Nav.Fac.Engr.Cen. 

1 ,500 

300 

12  PC-14 

Army  Missile  Com. 

1,200 

10,000 

12 

REClih  11 

Perry  Ocoanog. 

1,500 

450 

13  STAR  1! 

Deepwater  Expl . Ltd 

.1,200 

10,000 

13 

CORO 

Harbor  Br.  Found 

1 ,500 

770 

14  MERMAID 

Int'l  U.W.  Contr. 

1,000 

14,000 

14 

UARS 

Untv.  of  Washington 

1,500 

900 

15  MOANA 

COMEX 

1,300 

20,000 

15 

SCAT 

Nav.  Ocean  Sys.  Ctr. 

2,000 

400 

16  SEA  OTTER 

Arctic  Marine 

1 ,100 

6,300 

16 

CON SUB 

inst.of  Geology, U.K. 

2,000 

1,760 

17  DLEPVIEW 

SW  Research  Inst. 

1 ,500 

12.000 

17 

OEEPDRONC 

AmeteL  Strata 

2,000 

5,000 

18  PISCES  I 

Vickers  Ocrenics 

1.500 

15.000 

18 

RUFAS  II 

Miss.  State  Univ. 

2,400 

1,000 

19  PC-9 

P A 0 Subsea 

1,350 

22.500 

19 

CURV  IIB 

Nav.  Torpedo  Sta. 

2.500 

3,000 

20  PC-17 

Perry  Oceanog. 

1 ,500 

38,000 

20 

CURV  II 

Nav.  Ocean  Sys.  Ctr. 

2,500 

3,450 

21  OEEPSTAR 

G.O.  Int'l. 

2.000 

15.000 

21 

SCORPIO 

Aii'eteK  Strata 

3,000 

1,500 

22  SEA  LINK  11 

Harbor  Br.  Found. 

2,000 

21 ,000 

22 

SKORrENA 

U.S.S.R. 

3,300 

1,000 

23  SDL-1 

Canadian  Navy 

2,000 

30,000 

23 

ERIC 

French  Navy 

3,300 

4,410 

24  BEAVER  IV 

Int'l.  V.W.  Contr. 

2,700 

34,000 

24 

WORK  VEH 

HYDROTECH 

4,000 

40,000 

25  AllG.PlCCnRD 

Horton  Maritime 

2.500 

366,000 

25 

VERT. TRANS. 

26  PC-16 

InterSub 

3,000 

33,000 

VEH. 

HVOROTECH 

4,000 

100,000 

27  PISCES  11 

Vickers  Oce-nics 

2,400 

24,000 

26 

RCV  225 

HYDROPROOUCTS 

6,600 

180 

28  PISCES  VI 11 

Vickers  Oceanics 

3.00D 

24,000 

27 

RCV  150 

HYOROPROOUCTS 

6,600 

1,000 

29  PISCES  lit 

Vickers  Oceanics 

3,000 

24,000 

28 

SORO  I 

Naval  Torpedo  Sta. 

6,500 

4,000 

30  OEEPSTAR 

29 

SORO  II 

Naval  Torpedo  Sta. 

6,500 

4,000 

4000 

COMEX 

4,000 

18,000 

30 

SCARAB 

A.T.&T.  Co. 

6,000 

5,000 

31  DSRV  I • 

U.S.  Navy 

5,000 

75,000 

31 

DOWS 

AmeteR  Strata 

6,000 

5,000 

32  D5RV  II 

U.S.  Navy 

6,500 

75,000 

32 

CURV  III 

Nav.  Ocean  Sys.  Ctr. 

7,000 

4,500 

33  DOWB 

G.M. 

6,500 

20.000 

33 

TROIKA 

DCAN,  Fr. 

7,220 

2,000 

34  PISCES  VI 

HYCO  Subsea 

6,600 

24,400 

34 

RUM/CRB 

Scripps  Inst. 

8,000 

24,000 

35  P I SC 65  IV 

Dept,  of  Environ. 

6,600 

24,100 

35 

NEDAR  I 

Assoc.  Marine  Ser, 

10,000 

2,400 

35  PISCES  V 

HYCO  Subsea 

6,600 

24,400 

36 

KRAB-1 

Acad.  Scl .,  USSR 

10,000 

1 ,000 

37  PISCES  IX 

HYCO  Subsea 

6.600 

24,400 

37 

SPURV 

Univ.  of  Wash. 

12,000 

1 ,000 

38  SEA  CUFF 

U.S.  Naw 

6.500 

42,000 

38 

DEEP  TOW 

Scripps  Inst. 

20,000 

324 

39  SEA  TURTLE 

U.S.  Navy 

6.500 

42,000 

39 

MI2AR  FISH 

Nav.  Res.  lab. 

20 ,000 

4,300 

40  DEEPQUEST 

Lockheed 

8,000 

115,000 

40 

UOOSS 

Jet  Prop.  Lab. 

20,000 

3,000 

41  ALVIN 

Woods  Hole  O.I. 

12.000 

32.000 

41 

SEA  DRONE  I 

Pre  Con  inc. 

20,000 

2,800 

42  TRIESTE 

U.S.  Navy 

20.000 

180,000 

42 

TELFPROBE 

Nav.  Oceanog.  Off. 

20,000 

3,500 

43  ARCH  IMF  OF 

CNEXO 

36,000 

122,000 

43 

RUWS 

Nav,  Ocean  sys . Ctr. 

20,000 

4,300 

44  PC- 12(01 ) 

InterSub 

1 ,000 

18,000 

44 

NEDAR  II 

Assoc.  Marine  Ser. 

25,000 

1 ,BOO 

45  PC-12(03) 

COMEX 

1,000 

18,000 

45 

SEA  PROBE 

Ocean  Search  Inc. 

10,000 

400,000 

46  SFALINK  ! 

Harbor  Branch  Found 

.1 ,000 

21 .000 

47  V0L-L1&L2 

Vickers 

1,200 

28,000 

48  PC-1 2(02) 

InterSub 

1,000 

33.000 

50  JIM 

Oceaneerlng  Int'l. 

1,300 

1 ,100 

2-2 


FIGURE  2.1  VEHICLE  WEIGHT  VS.  DEPTH.  Manned  submersibles  must  necessarily 
be  more  massive'- (and  more  costly)  the  deeper  they  go  because 
of  the  necessity  of  providing  one  atmosphere  and  minimum  space 
for  pilot  and  observer.  Theoretical  limits  are  shown  for 
4-,  6-  and  8-foot  spheres.  These  are  based  on  steel  (o  = 80,000 
psi , E - 30  x 10°  psi),  a ratio  of  1.5  for  collapse  depth  to 
operating  depth  and  a ratio  of  3 for  vehicle  weight  to 
pressure  vessel  weight  (Evans,  1969). 

The  weight  (and  cost)  of  an  unmanned  submersible  is  less  dependent 
on  depth  and  more  dependent  on  the  tasks  it  is  designed  for.  For 
example,  vehicles  for  sensing  (20,  27,  32)  are  light  compared 
to  those  for  pipe-line  repair  (24,  25)  or  mining  (45). 
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Though  cost  data  are  mostly  not  available  we  venture  that  weight  is 
probably  the  best  single  predictor  of  vehicle  cost. 

2.2  Future  Undersea  Vehicles 


Figure  2.2  portrays  an  important  three-way  classification  in  undersea  work 
vehicles  in  terms  of  whether  vehicles  are  manned  or  unmanned,  tethered  or  un- 
tethered. 

In  the  previous  chapter  we  commented  that,  as  work  vehicles  (i.e.,  for 
inspection  and  manipulation,  neglecting  personnel  rescue  or  tranfer  from 
undersea  habitats),  there  remain  only  minor  advantages  to  manned  submersibles 
if  the  communication  link  and  control  systems  function  well.  The  advantages 
are  rapidly  being  counterbalanced  by  disadvantages.  The  advantages  are  ster- 
eopsis  of  human  vision  at  close  range,  high  resolution  combined  with  a wide 
field  of  view,  and  the  ability  of  the  human  operator  naturally  to  change  his 
direction  of  view  while  maintaining  a sense  of  where  he  and  various  environmental 
objects  are  located  relative  to  one  another.  But  these  advantages  may  be 
counterbalanced  by  the  costs  of  the  man,  the  pressure  vessel  and  the  required 
life  support  equipment. 

Experiments  in  video  stereopsis  suggest  that  such  techniques  are  gradually 
becoming  practical,  Resolution  can  be  obtained  by  remotely  controlled  zoom 
lenses,  As  for  proprioception  (keeping  track  of  the  configuration  of  the 
remote  mechanical  arm)  this  is  known  to  be  an  important  drawback  of  present 
teleoperation  techniques;  but  there  are  promising  possibilities  for  "tele- 
proprioception" using  head  mounted  CRTs,  fiber  optics,  and  the  like,  and  using 
replica  controllers  and  local  models.  (This  is  discussed  further  below.) 

Thus,  as  the  prerequisites  of  communication  and  control  are  fulfilled,  the  func- 
tional advantages  of  the  manned  submersible  are  disappearing,  while  the  cost 
disadvantages  remain. 

One  further  motivation  for  building  so  many  manned  submersibles,  as  compared 
to  emphasizing  unmanned  teleoperation,  should  be  mentioned.  It  is  the  same 
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MANNED 


UN- MANNED 


FIGURE  2.2  VEHICLE  CATEGORIES  for  general  purpose  submersibles  car,  be  used 
for  describing  needed  future  developments. 

As  operations  approacn  greater  depth  the  increased  cost  and 
safety  requirements  suggest  UNMANNED  vehicles.  Problems  of 
reduced  mobility  due  to  tether  drag,  tangle  and  snare  suggest 
UNTETHERED  vehicles.  Most  undersea  operations  are  incomplete 
without  some  kind  of  manipulation  or  WORK,  for  example  search 
anc  recover,  select  and  sample,  inspect  and  repair. 

Tre  few  untethered  unmanned  vehicles  in  existence  are  only  search 
ana  survey  vehicles.  Significant  problems  of  1 ’Ini ted  communi- 
cation must  be  solved  to  provide  either  better  control  from  the 
surface  (teleoperation)  or  automatic  contro1  (robotics)  or  a 
combination  (supervisory  control). 


1 


motivating  factor  we  have  observed  in  the  space  program,  namely  the  natural 
desire  of  the  human  being  to  achieve  and  experience  actual  presence  in  new  places. 
As  with  space,  we  expect  that  in  time  teleoperation  will  have  both  the  economic 
and  the  functional  advantage  for  undersea  operations  below  a few  hundred  feet 
in  depth. 

Figure  2.3  portrays  a two-way  classification  of  unmanned  work-vehicles 
(the  two  right-most  configurations  in  the  previous  figure).  The  vertical  dif- 
ferentiation of  Figure  2.2  (tether  vs.  no-tether)  breaks  down  further  in 
Figure  2.3  into  tether  (which  can  support  communications  and  power)  vs.  sonic 
communications (with  the  wave  pattern  indicating  souno  messages  can  go  both 
ways)  vs.  no  communications  at  all  (a  purely  "robotic"  vehicle).  Figure  2.3 
differentiates  horizontally  on  the  basis  of  whether  or  not  there  is  an 
intermediary  vehicle  or  structure  - a "garage"  we  call  it  - which  can  serve 
several  functions: 

1.  it  can  serve  as  a terminus  for  a tether  and  avoid  loading  the  tele- 
operator with  mechanical  forces  due  to  surface  waves  on  the  support 
ship  and  to  ocean  currents  integrated  over  the  whole  tether; 

2.  it  can  store  energy  and  allow  a battery-powered  teleoperator  to  return 
to  get  powered-up; 

3.  it  can  serve  as  a communications  way-station  and  permit  higher  band- 
width sonic  communication  (because  of  short  distance)  with  the  tele- 
operator at  relatively  low  power. 


i 
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FIGURE  2.3  COMMUNICATION  ALTERNATIVES  for  unmanned  vehicles  will  be 

important  in  determining  the  trade-off  between  human  and  computer 
control.  The  particular  configuration  will,  of  course,  depend 
on  task  to  be  accomplished,  operating  depth,  size,  speed, 
power  source,  duration,  etc.  The  above  matrix  classifies 
alternative  forms  of  communication:  1)  with  the  surface  ship 
(if  any);  2)  with  an  intermediary  "garage"  (if  any). 


3.  UNDERSEA  TASKS 


3.1  Classification  of  Tasks 

Teleoperators  are  used  for  inspection  and  manipulation  of  the  undersea 
environment.  Inspection  can  be  of  anything  that  a sensor  can  sense  and  for 
which  some  mobility  is  necessary  for  bringing  the  sensor  to  the  appropriate 
location  and  searching.  In  addition  to  the  obvious  need  to  see  and  locate 
(e.g.,  through  sonar)  natural  and  man-made  objects  many  other  environmental 
properties  are  of  interest.  Talkington  (1976)  presents  an  interesting  list 
of  such  properties  which  suggests  the  ocean  depths  at  which  such  sensing  is 
appropriate  (Table  3.1). 

Perhaps  it  is  possible  to  be  somewhat  more  specific  with  respect  to  mani- 
pulation tasks.  Drenning  (undat,)  out! ines  what  can  be  done  with  manipulators 
in  Table  3.2. 

All  of  these  tasks  must  be  accomplished  in  an  environment  that  at  times 
can  be  extremely  severe:  rough  seas  (an  air  water  interface  capable  of  de- 
molishing equipment  which  is  not  rugged),  corrosive  salt  water,  ocean  currents 
of  up  to  several  knots  which  tug  on  any  tether,  gale  winds  blowing  on  a surface 
vessel,  turbid  water  which  is  practically  opaque  to  light,  and  a rocky  and 
uneven  bottom. 

3 • 2 Task-Tool  Matching 

The  design  and  control  of  teleoperators  for  undersea  tasks  obviously  depends 
upon  what  those  tasks  are.  This  is  true  even  though  teleoperators  are  claimed 
to  be  general  purpose  inspection  or  manipulation  devices.  The  point  is  that 
any  one  teleoperator  is  relatively  general  purpose  within  some  necessarily 
limited  range  of  capabilities,  A teleoperator  suited  for  heavy  construction 
is  not  the  best  one  for  repairing  a watch. 

As  with  any  task  and  any  tool  or  instrument  for  accomplishing  that  task, 
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TABLE  3,1  OCEAN  EXPLORATION  AND  SURVEY  PARAMETERS.  (Talkington,  1976) 


Air-Sea  Upper  Water  Lower  Water 

Interface  Column  Column 

Parameter  ( IU  to  -10  m)  (-10  m to  -500  in)  (-500  m atul  deeper)  Bottom  Subbottom 


1 lee  X 

2 Sca-swell-surf  X 

3 Surface  meteorology  X 

4 Surge  X 

5 Tides  X 

6 Currents  X 

7 Hydrodynamic  forces  X 

K Noise  X 

0 Salinity  X 

10  Temperature  X 

11  Tuibidity  X 

I 2 Biomass  X 

13  Nutrients  X 

14  Oxygen  X 

15  Pollutants  X 

10.  Llectucal 
17  Bathymetry 
IS  Geomorphology 
ll>  Rheology 


20  l-.ngiiiceiing  piopeiUcs 

21  Geochemist  ly 

22  Geology 

23  Geothermal 

24  Physical  proper  tics 

25  kadiometlic 
2h  Gravity 

7.7  Magnetics 
2X  Seismic 


X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 


X 

X 

X 


X 


X 

X 

X X 

X X 


X 

X 


X 


X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 


X 

X 

X 

X 

X 

X 

X 

X 

X 
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Salvage 

Detach  cables  restraining  objects  to  be  salvaged 
Clear  debris  away  from  objects  to  be  salvaged 
Prepare  object  for  lifting  by  attaching  cables 
Position  objects  for  salvage 
Separate  large  objects 
Excavate  bottom  sediment 
Undersea  Rescue 

Aid  in  freeing  entrapped  submersibles 
Aid  in  mating  of  rescue  submersible  to  submarine 
Service  Habitats 

Aid  in  heavy  work  operations 
Aid  in  replenishment  of  supplies 
Aid  in  placement  and  recovery  of  habitats 
Offshore  Oil/Gas  Production  Faci.ities  Task 
Assist  during  drill  string  landing 
Prepare  drill  sites  by  removing  debris 
Replace  blowout  preventer  rams 
Make  pipe  connections 
Replace  and  patch  pipes 
Recover  objects  dropped  from  drill  platform 
Inspect  oil  lines  using  hand  held  acoustical  devices 
Remove  marine  growth 

Others 

Place  and  retrieve  acoustic  markers 

Place  explosive  devices 

Clear  and  remove  debris 

Collect  marine  samples 

Position  transponders 

Remove  and  replace  defective  equipment 

Take  bottom  core  samples 

Collect  mineral  laden  nodules 


TABLE  3.2  TYPES  OF  TASKS  A MANIPULATOR  CAN  PERFORM  ON  UNDERWATER  MISSIONS 
(Drenning,  undated) 
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teleoperators  and  undersea  tasks  must  be  matched  in  terms  of  a number  of  physi- 
cal variables  if  they  are  to  co-function.  The  classes  of  these  variables  are 
shown  in  Figure  3,1.  The  match  requires  that  the  range  of  each  variable  char- 
acteristic of  the  task  lie  within  the  operating  range  of  the  teleoperator. 


FIGURE  3.1  CLASSES  OF  VARIABLES  WITH  RESPECT  TO  WHICH  TASK  AND  TELEOPERATOR 
MUST  BE  MATCHED. 


3 , 3 Problems  of  Specifying  the  A ttributes  of  Under sea  T asks 

Specification  of  the  attributes,  the  characteristic  variables,  of  real 
undersea  tasks  can  never  be  precise.  This  is  for  two  reasons: 

1.  Different  operational  needs,  plus  different  equipments  by  different 
manufacturers,  plus  natural  technological  evolution  of  devices  and 
procedures,  make  the  tasks  a continuously  evolving  process. 
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2.  Those  who  have  analyzed  undersea  tasks  have  used  different  approaches. 

The  choice  of  task  analysis  technique  is  arbitrary. 

Our  approach  to  task  analysis  was  to  visit  and  talk  with  a number  of  govern- 
ment and  private  agencies  - undersea  operating  companies.  Naval  laboratories, 
etc,,  - and  review  the  diverse  literature  in  the  area.  From  this  we  have  dis- 
covered that  there  is  some  apparent  consensus,  though  the  approaches  to  analysis 
are  varied.  We  give  examples  below. 

3.4  Methods  of  Specifying  and  Analyzing  Undersea  Tasks 

At  a "mission"  level,  specification  of  the  task  is  done  by  maps  and  cross- 
sectional  diagrams  of  the  location,  terrain,  types  of  structures  or  cables  to 
work  with  or  around,  bottom  conditions  (i.e.,  soft  mud,  boulders,  etc.),  water 
turbidity  on  the  bottom,  prevailing  current  velocities,  surface  conditions  (ice, 
depending  on  location;  waves,  depending  on  season  and  weather). 

A typical  next  level  of  task  analysis  is  to  break  some  "overall  task"  or 
"mission"  into  smaller  elements  which  are  common  to  all  tasks.  This  common  set 
of  subtasks  is  comparable  to  the  "therblig"  set  developed  by  Lilian  Gilbreth 
(and  named  after  her,  only  spelled  backwards)  and  by  other  industrial  engineers 
to  characterize  common  task  elements  of  human  workers  on  the  production  line. 

Such  analysis  can  be  done  in  several  different  ways.  One  is  to  indicate 
whether  (and/or  how  often,  or  to  what  degree)  certain  subtasks  occur  within 
each  larger  task  or  mission,  Figure  3.2,  an  analysis  by  Bien  and  McDonough  (1971) 
is  an  example  of  this,  for  the  particular  case  of  human  divers  performing 
Naval  undersea  missions.  It  cross  plots  "performance  requirements"  with  type 
of  manual  task. 

A related  form  of  such  analysis  is  a time  line,  as  illustrated  in  Figure  3.3, 
(same  authors).  By  scaling  the  time  axis  as  a percentage  of  total  time,  one 
can  make  quick  judgements  about  where  most  of  the  time  is  spent.  Also,  one  can 
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FIGURE  3.2  NAVAI  UNDERSEA  MISSIONS,  OPERATIONS,  AND  ASSOCIATED  TASKS.  Shading 
indicates  that  task  occurs  in  broad  mission  area,  X indicates  that 
task  occurs  in  narrow  mission  catgegory.  (Bien  and  McDonough,  1971) 
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FIGURE  3.3  TASK-TIME  DISTRIBUTION  FOR  SHIP  SALVAGE  OPERATION. 

Shading  indicates  task  activity  for  associated  period  of  task, 
(Bien  and  McDonough,  1971) 
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see  what  subtask  elements  tend  to  go  together. 

By  integrating  over  all  subtask  elements  and  over  a variety  of  tasks  or 
missions,  one  can  produce  a pie-chart  or  comparable  representation  showing 
the  fraction  of  total  tasks  and/or  fraction  of  total  task  time  (which  are,  of 
course,  different!)  for  undersea  tasks. 

Figures  3,4  and  3,5,  replots  of  fraction-of-total-task  data  from  Ocean 
Systems,  Inc.  (1977)  show  such  data.  Figure  3.4  relates  force  and  accuracy.  The 
important  message  of  Figure  3.4  is  that  80%  of  all  tasks  require  neither  high 
torque/ force  nor  fine  orientation  movements.  The  important  message  of  Figure 
3.5  is  that  31%  of  all  underseas  tasks  are  visual  inspection  tasks,  requiring 
no  manipulator  at  all,  while  22%  (according  to  this  analysis)  are  involved 
with  removing  and  installing  flexible  wires  and  slings.  It  is  important  to 
keep  in  mind  that  in  such  task  analyses  the  quantitative  ranges  of  categories 
(e.g.,  on  force)  are  arbitrary  and  the  assignment  of  cases  to  such  categories 
surely  involves  a great  deal  of  subjective  judgement.  Nevertheless,  such 
analyses  provide  useful,  if  coarse,  information  to  the  designer  or  operational 
decision-maker, 

Figure  3.6  (from  Pesch,  Klepser  et  al . , 1970)  illustrates  the  complementary 
plot,  fraction  of  total  time  for  different  work  segments,  in  this  case  for  the 
task  of  recovering  five  50  lb.  lead  samples  from  the  ocean  floor. 

In  our  own  laboratory  Schneider  (1977)  performed  a task  analysis  in 
which  he  broke  what  might  be  considered  mission  subtasks  (lefthand  column  of 
Table  3.3)  into  two  lists:  general  work  tasks  and  tasks  specialized  to  the 
oil  industry.  Then  he  made  a further  breakdown  according  to:  forces  (how 
much  indicated  in  parentheses)  in  rotation  or  translation,  whether  they  are 
constant  or  mpact  forces;  placements  or  positioned  movements,  qualified  by 
what  kind;  v.hat  mode  of  sensory  feedback  to  the  human  operator  is  salient; 
and  finally  some  statistical  attributes  of  the  situation  geometry  and  some 
factors  relating  to  whether  two  arms  and/or  special  tools  are  needed.  The 
numbers  1,  2,  3 represent  the  seque’ictng  of  steps  within  a task,  the  letters 


3-8 


Trsmm! 


PIVEKS1  TASi'CS  - drilling  gg^aort 


1493  Dives 

2.725  TASKS 


INftfECr  BEALS 
rie/orr  softumc 
MEASURE  WITH  IM6TK 

«5m/i hst;  5upRi*r  eaotf! 
KeM/nWT-  Nure/*°I-Ts, 
lMSRecr  all 
COLLECT  BAMR 
cur  ALL  — 

CLEAR  All 


REH/IMC-T. 
MypKAUUO© 


GUIPC  eifi 

6VIPC  STAB 
SEARCH 

MEASURE 


o&sekve 

COMPITJCNS 


STA6  ALL 


R6M0VE/IH6WLL 
GuloeW(«5S/‘SUH66/TU66feKS 


RCTURNB 

BTAB/LAMPlNfr 

Bottom 

ewipewiRE/tfaope 

LEALS 

LATCH  INO  P1MS 

wcllhead/riseij/rm> 


I FIGURE  3.4  DIVERS'  TASKS.  Ocean  Systems,  Inc.,  (1977),  a diving  service 

| y company,  compiled  statistics  on  what  tasks  their  divers  per- 

■ J formed  on  1493  separate  dives  in  support  of  undersea  drilling 

[ operations.  Observation  accounted  for  only  31%  of  the  tasks; 

f all  the  rest  involved  some  form  of  manipulation.  The  various 

| l actions  are  categorized  in  Figure  3.5  according  to  force  and 

| •*  dexterity  required. 
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FIGURE  3.5  DIVERS’  ACTIONS  were  classified  by  Ocean  Systems.  Inc., 

(1977)  into  nine  classes  depending  on  the  force  and  dexterity 
required.  For  example,  the  heavy  force  was  any  action  requiring 
a pull  of  over  50  lbs.  The  three  classes  of  manipulative 
dexterity  were  gross  positioning  such  as  swimming  and  dropping 
(37%),  accurate  positioning  relative  to  a fixed  point  such  as 
with  a crane  (30%),  and  accurate  positioning  and  orientation  at 
a fixed  point  such  as  the  diver's  wrist  provides  (33%). 

An  unmanned  "diverless  support  system"  might  not  be  able  to  do 
the  highest-dexteri ty-lowest-force  actions  or  any  of  the  high- 
force  actions  but  that  leaves  77%  of  all  tasks.  "In  combination 
with  special  tools,  manipulators  could  perform  almost  all  divers' 
tasks . " 
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P«rceat  ot  Total  Task  Time 


M and  L indicate  that  "medium"  or  "long"  (occasional)  update  periods  between 
visual  samples  will  do,  as  compared  to  continuous  update,  C. 

Given  Schneider's  results,  it  is  evident  that  translational  force  requirements 
seldom  exceed  50  lbs.,  that  vision  contributes  to  every  task,  that  gross  force 
(as  might  be  felt  by  a person  in  the  muscles  and  tendons,  and  is  built  into  force- 
reflecting  master-slave  manipulators)  is  next  most  important.  Special  tools 
seem  to  be  needed  for  about  half  the  tasks. 

Figure  3.7  characterizes  task  requirements  of  special  end-effector  tools, 
which  typically  are  attached  on  to  the  end  of  the  arm  in  place  of  the  grippers 
and  driven  hydraulically.  Naturally,  these  special  tools  are  needed  because 
the  grippers  will  not  perform  the  required  actions.  Figure  3.7  describes  the 
end-effector  movement  requirements  by  "adverbs"  (different  columns)  indicating 
how  or  what  kind  of  action  (row)  is  necessary. 

A quite  general  way  to  characterize  a particular  task  is  in  terms  of  a 
flow  chart,  such  as  is  used  tc  characterize  a computer  program  (which,  of 
course,  is  a description  of  an  information  processing  task).  Figure  3.8 
gives  an  example  of  such  a flow  chare  applied  to  a hypothetical  oceanographic 
sample  collection  task.  Rectangles  represent  actions,  diamonds  represent 
decisions.  Such  diagrams  allow  as  detailed  a specification  of  a task  as 
the  analyst  has  patience  for.  This  type  of  analysis  specifies  where  measure- 
ments must  be  made,  where  controlled  actions  must  be  taken,  and  generally  deals 
with  the  kind  of  infonnat.ion  necessary  either  for  programming  a computer 
or  for  teaching  an  operator  to  du  a task. 

A close  cousin  to  the  flow  chart  is  the  PERT  chart,  or  time  precedence 
diagram.  Figure  C.9  illustrates  the  iaea  using  a similar  hypothetical  task. 

The  special  significance  of  this  diagram  for  task  analysis  is  that  it  tells 
what  must  be  done  before  what  else,  or  when  it  doesn't  matter  which  goes 
first.  It  is  a useful  tool  fr*"  planning  a mission  and  predicting  how  long  it 
wi 1 1 take . 

When  there  is  a relatively  comman  s<.  i of  elements,  one  may  do  a frequency 
count  on  transitions  between  tisk  elements,  and  summarize  the  analysis  by  a 
transition  diagram,  Figure  3.10.  The  elements  between  which  the  frequency  is 
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FIGURE  3.7  END  EFFECTOR  TOOL  MOTIONS  FOR  SALVAGE  TASKS.  (Battel  1 e , 1976  ) 
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highest  should  be  moved  in  closest  proximity  or  most  facilitated  to  mini- 
mize time  and  errors. 

There  are  many  other  ways  to  analyze  undersea  tasks  , but  the  examples 
cited  above  provide  a cross-section. 

3 . 5 Tasks  in  the  Laboratory  Environment  vs.  Tasks  in  the  Real  World 

In  designing,  building  and  testing  teleoperators,  and  in  trying  to  under- 
stand the  nature  of  actual  undersea  tasks,  it  is  useful  to  devise  experimentally 
controlled  laboratory  tasks.  Essentially  , with  the  laboratory  task  it  is 
not  the  accomplishment  of  the  task  per  se  which  is  important,  but  the  manner  of 
doing  the  task  - either  as  a demonstration  that  a particular  teleoperator  can 
do  a particular  task,  or  as  a measure  of  how  quickly  a fixed-accuracy  task 
can  be  done,  how  accurately  a fixed-time  (or  open  ended)  task  can  be  done,  etc. 

The  laboratory  task  can  be  repeated,  so  that  reliability  measurements  are 
possible.  The  laboratory  task  can  be  scaled,  so  that  the  same  basic  task  can  be 
posed  but  with  different  tolerances,  or  sizes,  or  orientations,  etc. 

Laboratory  tasks  may  be  differentiated  with  respect  to  the  following  object- 
ives : 

(1)  The  laboratory  task  may  be  intended  as  a simulation  of  a real  task,  so  that 
as  many  elements  of  realism  as  possible  and  practical  are  brought  into  the  simu- 
lation. Alternatively  (2),  laboratory  tasks  may  be  everyday  manipulation/ 
inspection  task,  which  have  the  attractive  attribute  that  people  have  some 
experience  with  them  , understand  them,  and  have  some  expectations  about  how 
they  can  or  should  be  done.  Or  (3),  laboratory  tasks  may  be  "cal ibration  tasks" , 
adjustable  in  quantifiable  ways  such  that  objective  numerical  scores  may 
easily  be  obtained.  Finally  (4),  laboratory  tasks  may  be  archetypal , selected 
primarily  to  "show-off"  a particular  teleoperator  to  best  advantage. 

Figure  3.11,  a typical  peg-in-hole  laboratory  task  employed  by  Hill  0977), 
illustrates  a simple  means  to  calibrate  both  distance,  force,  and  tolerance 
(between  peg  and  hole).  Figure  3.12  illustrates  a set  of  tasks  used  by  Hill  which 
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differ  from  one  another  in  terms  of  number  of  degrees  of  freedom  which  must  be 
controlled. 

The  following  set  of  seven  task-teleoperator  categories  is  ordered  from 
fully  contrived  laboratory  environment  to  fully  "real  world"  environment. 

Between  3 and  4 the  shift  is  made  from  air  to  artificial  or  natural 
bodies  of  water. 

Laboratory  environments: 

1.  Fully  contrived  archetypal  tasks  - explicitly  chosen  to  characterize  "the 

best  a given  teleoperator  can  do",  i.e.,  the  most  accurate,  or  fast,  or 
complex,  etc.  task  the  teleoperator  can  perform  with  a trained  operator 

- a different  set  for  each  teleoperator 

2.  Calibration  tasks  - designed  to  yield  objective  evaluation  measurements 

- standardized  battery  of  tasks  accepted  within  the  community  of  interest 

- task  and  time  constraints  well  specified  and  communicated  to  operator, 
including  time-error  tradeoff  criteria  according  to  which  he  will  be  scored 

- operator  usually  trained  with  teleoperator  equipment  to  be  used,  which 
may  be  operational,  developmental,  or  for  research 

- test  usually  repeated  over  a factorial  array  of  task  parameters,  criteria 
constraints  or  operators 

3.  "Everyday"  manipulation/inspection  tasks  - usually  performed  as  demonstrations 

to  give  observers  some  intuitive  sense  of  what  teleoperator  can  do  by  com- 
parison to  people.  Examples  are  stacking  blocks,  putting  nuts  on  bolts, 
tying  knots  in  ropes,  lighting  cigarettes,  writing  name  with  pencil,  etc. 

- same  conditions  apply  as  in  2 

Underwater  Environments 

4.  Simulated  "real  undersea  tasks"  - usually  for  demonstration 

- time  and  risk  constraints  artificially  imposed  on  operator 

- task  well  known  to  experimenter,  may  not  be  to  operator 

- teleoperator  equipment  may  be  operational  or  developmental 

- operator  usually  chosen  to  do  best  possible  job 

- may  be  in  water  or  in  air 
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5.  Checkout  in  protected  harbors 

- mild  time  stress 

- tasks  well  defined;  but  turbidity,  depth,  etc.  may  be  different  from 
operational  condi tjdns 

- teleoperator,  support  equipment  and  operators  carefully  selected 

- hazards  due  to  nature  removed 

6.  Sea  trials 

- moderate  time  stress 

- some  uncertainties  concerning  task 

- teleoperator,  support  equipment  and  operators  selected  for  test 

- sometimes  hazards  due  to  nature,  usually  none  due  to  enemy 

7.  Operational  mission  situations 

- severe  time  stress 

- uncertainties  of  what  the  task  is 

- available  teleoperator  and  support  equipment  not  necessarily  the  most 
appropriate 

- available  operators  not  necessarily  the  best  trained 

- hazards  due  to  nature  and/or  enemy 

3 . 6 Formal  Specification  of  a Manipulation  Task  in  N-dimensional  State  Space 

Given  all  the  variables  which  must  be  controlled,  given  the  tolerances  to 
which  they  must  be  controlled  relative  to  other  variables,  and  given  the  order 
in  which  these  control  actions  must  take  place,  a formal  description  of  a 
manipulation  task  can  be  couched  in  terms  of  a "tunnel  through  state  space". 

By  letting  each  variable  be  a different  orthogonal  axis  in  Cartesian  space, 
including  time  or  higher  time-derivatives,  we  may  represent  every  possible  con- 
figuration of  objects  to  be  manipulated  relative  to  the  environment  as  a single 
point  in  this  multidimensional  space.  A connected  sequence  of  such  points  is  a 
precise  specification  of  how  the  task  was  done,  or  ought  to  be  done,  etc.,  An 
example  from  Whitney  (1969a)  is  shown  in  Figure  3.13. 

If  each  such  trajectory  point  is  surrounded  by  a hypercube  which  specifies 
tolerances  for  each  variable  at  that  point  in  the  task,  then  the  connection  of 
all  those  hypercubes  represents  a "tunnel"  in  state  space.  Such  formal  repre- 
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FIGURE  3.13  A SIMPLE  TASK  AND  ITS  STATE  GRAPH.  The  upper  figure  represents 
a physically  two-dimensional  task  with  two  moveable  objects. 

Jaws  can  be  opened  or  closed  (one  dimension  of  state  space), 
and/or  moved  to  one  of  four  numbered  positions  (second  dimension 
of  state  space).  Block  can  also  be  moved  to  one  of  same  four 
positions  (third  dimension  of  state  space).  Diagonal  arrows 
represent  "pushing"  from  either  left  or  right.  The  dashed  line 
started  at  the  circle  indicates  a trajectory  required  to  grasp 
the  block  at  2,  move  it  to  3,  release  it  there,  get  behind  it 
and  push  it  to  4. 
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sentations  of  tasks  may  be  useful  for  theoretical  purposes,  such  as  planning 
of  computer  programs.  The  difficulty  with  the  state-space  representaton  is 
that  a "real"  task  usually  involves  sufficiently  many  dimensions  that  it  is 
usually  too  unwieldy  for  a computer  to  handle,  or  a human  mind  to  specify 
or  even  comprehend, since  the  number  of  states  to  be  stored  is  the  product  of 
the  number  of  variables  and  the  number  of  states  per  variable.  Optimization 
requires  that  all  possible  sequences  of  all  states  be  considered.  Thus,  as 
the  task  complexity  increases  the  optimization  problem  rapidly  gets  out  of 
hand. 


What  are  needed  are  relatively  simple  indices  or  relationships  between 
task  variables  which  are  predictive  of  task  performance.  There  are  very  few 
examples  to  cite,  but  one  very  useful  such  index  is  that  attributed  to  P.  M.  Fitts, 
sometimes  called  the  "index  of  difficulty"  and  equal  to  the  log  of  the  ratio 
of  required  move  distance  to  required  tolerance  of  final  position.  Examples 
of  the  use  of  Fitts'  index  will  be  cited  in  Section  7. 
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; 4.  MANUAL/COMPUTER  CONTROL 

i 

4.1  Nodes  of  Servomechanism 

Control  means  to  make  a thing  do  what  is  desired.  There  are  two  main 
problems  in  control: 

1)  to  decide  what  is  desired; 

2)  to  make  the  thing  do  it. 

With  respect  to  the  second  problem,  we  may  illustrate  some  common  distinc- 
tions between  modes  of  control  by  reference  to  Figure  4.1,  which  models  a 
manipulator  system  as  a conventional  feedback  control  system  or  "servomech- 
anism". If  all  of  the  feedback  loops  are  closed  continuously  this  is  a 
force-reflecting  master-slave  position  servo-mechanism. 

Not  all  of  the  feedback  loops  need  be  closed.  For  example,  assuming  the 
human  operator  has  an  adequate  visual  feedback  channel  (y^  y,,  in  terms  of 
spatial  resolution  and  video  frame  rate,  satisfactory  control  of  a manipulator 
or  vehicle  or  sensor  position  (x)  may  be  achieved  by  the  human  operator  by 
comparing  what  is  observed  to  what  is  intended  (r),  deciding  on  the  basis  of 
this  position  discrepancy  (e)  what  to  ask  (u^)  his  muscle  to  send  (u?)  to  the 
joystick  or  other  hand  controls,  which  in  turn  communicate  across  a channel 
(u^  to  u4)  directly  to  an  actuator,  such  as  a hydraulic  valve-cylinder  device, 
which  finally  exerts  a force  (u^)  against  a load  or  disturbance  (v),  the 
difference  between  which  (u^)  drives  a mechanical  device  to  a position  (x), 
depending  upon  its  dynamics.  In  this  case,  assuming  the  position  servo  signal 
(yQ)  and  the  force  feedback  channel  (z1  z^)  don't  exist  we  have  pure  "rate 

control",  where  visual  position  feedback  is  the  sole  basis  for  achieving  even- 
tually a desired  position,  corrupted  of  course  by  disturbances  in  the  y^  y^  y 3 
or  U|  u^  u^  u^  channels.  In  this  case  the  teleoperator  may  be  said  to  be 
under  "open  loop  control". 

Another  form  of  open  loop  control  is  where  the  visual  feedback  loop  is 
closed  only  intermittently  and  during  a lapse  of  such  visual  feedback  the  operator 
commands  a u^  signal  which  he  estimates  to  be  appropriate.  In  this  case,  if 
position  feedback  y^  is  available,  the  measured  descrepancy  between  it  and 
| **  command  u^  drives  x into  conformity  with  u^. 
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If  the  forces  applied  to  the  terminal  device,  and  sensed  either  by  strain 
gages , or  by  the  armatire  current  in  the  motor,  are  fed  back  to  the  operator's 
hand  (through  the  z1  z2  z3  channel,  possibly  modified  in  scale,  hopefully  not 
biased  relative  to  zero)  we  speak  of  "force  feedback".  Such  force  feedback  can 
be  used  with  rate  control  (i.e,,  without  position  feedback  y^)  or  without  visual 
feedback  (y-j  y^  y3).  Excluding  both  visual  and  artifically  sensed  position  (Yq) 
feedback  simultaneously  would  probably  not  be  satisfactory  except  in  certain 
cases  where  "pure  force  control"  is  desired,  independent  of  position. 

If  force  and  both  types  of  position  feedback  are  available  simultaneously, 
we  have  a conventional  "force-reflecting  master-slave  position  servo"  as  stated 
earlier.  The  term  "bilateral"  is  sometimes  appended  when  such  a system  is 
designed  symmetrically,  where  the  force  display  is  actually  a motor-driven  hand 
controller,  and  where  any  position  error  between  master  and  slave  forces  the 
slave  hand  in  one  direction  and  the  master  hand  controller  in  the  other.  In 
most  such  systems  either  end  can  serve  as  the  master,  the  other  the  slave. 

It  is  not  our  purpose  here  to  review  standard  control  systems  theory. 

Suffice  to  say  that  if  each  of  the  blocks  in  Figure  4,1  were  describable  by  a 
linear  differential  equation,  the  whole  system  becomes  amenable  to  analysis  by 
conventional  linear  control  theory  - stability  analysis,  and  so  on.  This  is 
true  even  if  each  of  the  variables  (lines)  is  a vector  of  many  dimensions  and 
the  blocks  are  matrices  of  linear  operators. 

An  "optimal"  determination  of  the  human  operator's  control  strategy  as  a 
servomechanism  can  be  determined  by  solving  simultaneously  t.he  set  of  equations 
specifying  all  the  physical  transfer-functions  in  the  system  together  with  an 
equation  specifying  the  optimal  tradeoff  between  all  relevant  performance  varia- 
bles, e.g.,  x,  e , time,  and  energy  used.  In  practice  this  is  usually  easier 
said  than  done.  However,  if  the  element  transfer-functions  are  linear,  the 
tradeoff  criterion  is  quadratic,  and  the  disturbance  time  functions  are  Gaussian, 
it  is  straight  forward,  though  complex.  This  is  the  subject  of  so-called 
"modern  control  theory". 
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Even  if  we  have  a perfect  servo*,  i.e.,  x follows  r instantly  and  faith- 
fully with  insignificant  energy  or  other  cost,  the  major  problem  remains  that 
of  deciding  what  is  desired  - deciding  what  motions  to  execute  in  what  order. 

Deciding  what  to  do  in  what  order  is  something  people  appear  to  be  good  at. 

It  comes  close  to  what  is  considered  "motor  skill"  or  even  "intelligence". 

This  is  the  reason  that  programming  manipulators  to  do  clever  things  is  popular 
with  the  artificial  intelligence  conminity. 

4.2  The  Roles  of  the  Computer  in  Teleoperator  Control 

It  may  seem,  then,  that  if  a human  operator  were  available  to  decide  what 
motions  of  manipulator,  vehicle  or  sensor  need  to  be  executed  in  what  order,  then 
technological  effort  need  only  be  concentrated  on  making  servomechanisms  closer 
and  closer  to  perfection.  This  is  not  the  case,  or  at  least  the  situation  is 
not  quite  that  simple,  and  at  least  one  other  form  of  technology  which  is  not 
considered  to  be  part  of  the  servomechanism  per  se,  namely  the  computer,  has 
very  great  promise. 

A perfect  servomechanism  necessarily  would  have  to  include  instantaneously 
responding  actuators  with  no  dynamic  lags  or  static  distortions,  as  well  as  a 
perfect  visual  feedback  channel  y^  y^  y^  and  a perfect  force  feedback;  channel 
Z1  z2  z3*  the  degree  t*iat  suctl  motor  and  sensory  channels  approach  perfection, 
to  that  degree  the  teleoperator  system  will  become  transparent  to  the  human 
operator;  he  will  see  and  feel  himself  to  be  controlling  the  manipulator  as 
though  he  had  a pair  of  pliers  in  his  hand.  Perfect  manipulator  design  might 
further  demand  that  the  pliers  be  transparent  - that  the  operator  would  then 
sec  and  feel  as  though  he  were  interacting  with  environmental  objects  with  his 
bare  hands. 

A "perfect  handling"  vehicle  control  servomechanism  would  also  demand 
extremely  fast  actuators  and  a high  degree  of  both  visual  and  force  feedback 
and  so  too  would  that  for  positioning  a remote  sensor.  In  this  case  it  is  a 
bit  more  difficult  to  interpret  what  transparency  would  mean. 
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In  any  case,  to  achieve  such  transparency  - to  make  it  seem  to  the  opera- 
tor  that  he  is  present  at  the  remote  site  with  no  apparatus  intervening  between 
himself  and  the  end  object  of  his  control  - in  practice  is  not  attainable.  The 
computer  can  compensate  for  descrepancies  in  motor  response  and  feedback  in 
various  ways.  We  might  portray  these  computer  roles  in  four  categories,  as 
illustrated  in  Figure  4.2: 

1)  the  computer  can  also  compensate  for  less-than-perfect  decision-making 
or  other  capabilities;  it  can  extend  his  capabilities  to  help  the 
teleoperator  accomplish  more  than  if  he  alone  were  in  control. 

2)  it  can  relieve  him  of  some  control  tasks  while  he  concentrates  on 
others. 

3)  if  video  feedback  is  lost  for  brief  periods  the  computer  can  provide 
back-up  by  taking  ver  control. 

4)  it  can  replace  him  when  a task  can  be  programmed  and  is  too  dull  or 
fatiguing  to  warrant  his  continued  attention. 

It  is  interesting  to  consider  a continuum  along  which  the  "degree  of  auto- 
mation" can  vary  fromnone  (direct  manual  control  by  person)  to  complete  (hypo- 
thetical intelligent  robot  with  no  intervention  by  person).  The  collaborations 
between  human  operator  and  computer  we  are  considering  obviously  fall  part-way 
along  this  continuum.  Now  we  can  make  a qualitative  plot  (Figure  4.3)  of  the 
kinds  of  tasks  each  mix  of  human  and  computer  control  would  be  capable  of, 
measured  in  terms  of  unpredictability  or  "entropy"  (for  example,  in  simple  cases 
measured  in  information  theory  terms,  using  log  signal-to-noise  indices  such  as 
Fitts'  law). 

It  is  instructive  to  consider  alternate  design  philosophies  or  "development 
trajectories"  in  terms  of  the  two  variables  of  Figure  4,3  replotted  in  Figure 
4.4.  The  computer's  function  can  be  seen  to  compensate  for  feedback  deficiencies 
in  several  alternate  ways:  In  the  (a)  part  of  Figure  4.4  are  shown  three  develop- 
ment trajectories,  or  avenues  of  improvement  of  teleoperators  (ability  to  do 
more  complex  tasks).  The  trajectory  labeled  X shares  computer  control  with 
the  human  operator  to  improve  the  quality  of  the  continuous  sensory  feedback  or 
motor  feedforward;  here  essentially  all  control  decisions  are  made  by  the  human 
operator,  see  also  diagram  (b)  below.  Hence  X shows  little  increase  in  control 
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FIGURE  4.2  ROLES  OF  COMPUTER  in  supervisory  control  can  be  classified  according 
to  how  much  task-load  is  carried  compared  to  what  the  human  operator 
alone  can  carry.  The  computer  can  EXTEND  the  human’s  capabilities 
beyond  what  he  can  achieve  alone,  it  can  partially  RELIEVE  the  human, 
making  his  job  easier,  it  can  BACK-UP  the  operator  in  case  he  falters, 
and  it  can  REPLACE  him  completely. 

In  the  case  where  both  computer  and  human  are  working  on  the  same  task 
at  the  same  time,  we  call  this  SHARING  control.  When  they  work  on  the 
same  task  at  different  times  this  is  TRADING  control.  Different  modes 
create  very  different  demands  on  the  human  operator.  (See  Section  6). 
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FIGURE  4.4 


ALTERNATIVE  DESIGN  PHILOSOPHIES  FOR  TELEOPERATORS.  For  X 
control  is  mostly  shared  and  autonomy  increase  is  slight. 

For  Y control  is  mostly  traded  and  autonomy  increase  is 
large.  For  Z autonomy  is  added  with  no  improvement  in  task 
capability.  Design  (b)  shows  man  and  computer  collaborating 
in  sharing  mode,  (c)  shows  control  traded  to  computer, 
man  acting  as  supervisor. 
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autonomy  as  task  capability  increases.  This  kind  of  improvement  is  called  for 
in  situations  whore  there  is  no  time  delay,  but  visual  and  proprioceptive  feed- 
back is  of  poor  quality.  This  is  discussed  at  greater  length  in  Section  6. 

Trajectory  Y trades  control  between  computer  and  human  operator,  thereby 
gaining  significant  task  capability  when  there  is  a time  delay  or  visual  drop- 
out or  for  other  reasons  the  operator  cannot  function  as  an  effective  continuous 
controller.  This  is  the  mode  in  which  we  have  described  supervisory  control 
previously.  Diagram  (c)  represents  such  intermittency  of  control  through  the 
human  operator  to  the  computer,  while  the  computer  mediates  continuous  control 
with  the  human,  Along  such  a development  trajectory  the  gain  in  task  capability 
correlates  directly  with  increase  in  control  autonomy. 

Trajectory  Z shows  increase  in  control  autonomy  without  any  increase  in  task 
capability.  This  corresponds  to  situations  (also  represented  by  diagram  (c))  such 
as  record-playback  or  other  direct  automation  of  human  function,  simply  to 
back-up  or  replace  the  human  operator  in  a task  he  can  do  by  himself. 

A more  comprehensive  block  diagram  of  functions  performed  by  the  computer 
in  relation  to  the  operator  and  various  physical  elements  of  the  system  is 
given  by  Figure  4.5.  The  paths  through  the  central  "executive"  block  indicate 
the  information  source  and  destination  for  each  processing  function.  Note  the 
symmetry  of  sensors  for  effectors  and  effectors  for  sensors. 

4 . 3 The  Roles  of  the  Human  Operator  in  Supervisory  Control  of  a Teleoperator 

In  such  a supervisory  control  situation  the  human  operator,  like  the  computer, 
performs  in  different  roles  at  different  times,  Figure  4.6-  These  are  grouped  into 
four  role  categories. 

1.  Command:  to  program  a series  of  teleoperator  sensing,  mobility,  or  manipu- 
lation operations  and  commit  these  to  action.  These  commands  may  be  analogic 
(control  forces  and  displacements  which  are  geometrically  isomorphic  with  intended 
system  response)  or  symbol ic  (strings  of  alphanumeric  keystrokes  when  strung 
together  specify  unique  instructions).  Either  type  of  command  may  be  directed 
toward  teleoperator  movements,  or,  short  of  that,  toward  computer  executions  of 
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displays  of  desired  information  in  desired  formats,  or  of  program  changes. 

The  computer  may  help  by  checking  out  symbolic  commands  to  make  sure  they  are 
understandable.  (The  principal  concerns  for  engineering  this  role  are  what 
combinations  of  analog  and  symbolic  commands  are  best  for  what  kinds  of  tasks, 
and  what  kinds  of  feedback  signals  from  the  computer  will  most  help  the  opera- 
tor compose  appropriate  instructions  quickly  and  without  undue  "hassle"). 

2.  Plan:  to  consider  and  evaluate  various  alternative  future  commands  (for 
teleoperator  actions).  As  part  of  the  planning  process  the  operator  may  try 
these  out  by  commanding  a computer-based  model  which  implements  the  commands 
as  "thought  experiments"  off-line  in  fast-time.  The  results,  suggesting  "what 
would  happen  if",  can  be  displayed  side  by  side  on  a graphics  terminal  for 
the  human  operator  to  compare.  Non-computer  models  can  also  be  used  for  such 
planning  purposes  suc^  as  maps,  or  three  dimensional  scaled-down  models  of 
the  vehicle  or  manipulator  or  environmental  objects  which  may  be  moved  in 
relation  to  one  another.  Sometimes  computer-based  planning  models  and  displays 
can  be  utilized  simultaneously  with  control  of  the  teleoperator;  a pndictor 
display  (discussed  later)  is  an  example.  (Our  principal  engineering  concerns 

for  this  role  are  what  kinds  of  models  are  most  useful  to  the  operator's  planning, 
and  what  time-span  or  cycle-time  of  planning  and  action  are  best,  again  both 
depending  upon  task) . 

3.  Monitor:  to  observe  various  displays  of  teleoperator  performance  while 
the  latter  is  automatically  implementing  commanded  actions;  to  adjust  the 
displays  to  provide  different  presentations  of  data  to  be  monitored;  to  make 
small  parametric  adjustments  in  control  parameters;  to  instruct  the  computer  to 
make  on-line  diagnoses.  (Here  we  are  concerned  with  how  much  and  what  detail 
to  give  the  operator  to  help  him  keep  track  of  the  key  variables.  We  are  also 
concerned  with  problems  of  mental  overload  and  underload.) 

4.  Intervene:  to  take  over  control  manually;  to  change  control  to  a mode  more 
manually  direct  than  would  normally  be  the  case;  to  cause  the  computer  to  shift 
into  a preprogrammed  "abort"  mode;  or  to  stop  the  teleoperator  action  altogether. 
(For  this  role  the  concern  is  what  criteria  are  most  appropriate  for  the  inter- 
vention, what  the  transient  response  is  likely  to  be  under  various  task-tele- 
operator situations,  and  how  the  system  can  be  designed  to  be  "fail  soft"). 
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5.  Trust:  to  attain  enough  operating  experience  or  otherwise  acquire  a basis 
for  believing  that  the  teleoperator  will  behave  as  intended;  to  understand,  come 
Into  temporal  and  spatial  synchrony  with,  and  Identify  or  empathize  with  the 
responses  of  the  te'leoperator  to  various  commands.  (Here  the  concern  is  how  best 
to  help  the  operator  acquire  this  identification.  It  depends  on  selection, 
training,  and  the  naturalness  and  quality  of  feed-forward  and  feedback  loops.) 

Section  6 continues  the  discussion  of  supervisory  control  of  teleopera- 
tors. In  the  next  section  we  discuss  the  effects  of  hardware  configuration 
on  teleoperator  control  - in  particular  the  hardware  for  sensing,  communication, 
display,  vehicle  mobility,  manipulation,  and  command  interfacing  with  the 
operator. 
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5.  CONTROL  HARDWARE  FOR:  SENSING,  COMMUNICATION,  DISPLAY,  VEHICLE  MOBILITY, 
MANIPULATION,  COMMAND. 

5.1  Sensory  Systems  and  Their  Environmental  Constraints 
Optical  Sensing 

The  most  important  sensory  mode  for  teleoperation  is  optical,  since 
video  is  well  developed  and  directly  interpretable  by  vision,  which  in  turn 
is  by  far  the  richest  sensory  mode  of  the  human  organism,  Table  5.1  indicates 
the  most  important  physical  attributes  corresponding  to:  the  video  source 
objects,  the  communications  channel  (including  water,  video  or  sonic  imaging 
sensors,  electrical  channel  and  CRT  display),  the  display  (physical  stimulus 
impinging  on  the  retina),  and  the  corresponding  behavioral  (visual)  response. 

The  ocean  environment  poses  severe  constraints  on  vision.  Referring 
to  the  numbers  in  Table  5.1: 

1.  Illumination  decreases  as  a function  of  depth,  except  for  biolo- 
gical or  man-made  illumination. 

2.,  3.  Reflectance  and  contrast  of  adjacent  objects  decreases  as 
all  objects  which  remain  undersea  become  coated  with  the  same 
plant  matter  and  silt. 

4 Colors  all  become  similar  for  the  same  reason  as  2., 3. 

5.  Patterns  and  textures  similarly  become  one, 

6.  Range  and  orientation  (lateral  position  in  sensor  field)  are 
unknown  or  only  partially  known  at  the  time  of  visual  search, 

7.  Spatial  resolution  of  the  sensor  (and  channel)  is  usually  far  worse 
than  that  of  the  eyes. 

8.,  9.  Turbidity  (both  density  and  particle  size)  of  the  water  depends 
on  location,  but  typically  is  the  major  factor  in  reducing  visi- 
bility. Reduction  is  both  by  forward  scattering  (reducing  bright- 
ness contrast)  and  by  omni-directional  scattering  (small  particles 
selectively  scattering  and  reducing  blues  more  than  reds).  In 
non-turbid  water  the  reds  are  at  sorbed  more  quickly  than  the  blues 
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TABLE  5.1  CONSTRAINTS  ON  VISION 
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(yaughan  et^  al . , 1977). 

.10.  A computerized  signal  conditioning  can  enhance  contrast  and  Improve 

* » 

some  aspects  of  the  picture  hut  sometimes  at  a loss  in  gray-scale 
or  other  potential  information. 

11.  Frame  rate  is  usually  faster  than  the  eye^s  flicker-fusicn  fre- 
quency with  tethered  video,  but  may  reduce  to  one  frame  per  sev- 
eral seconds  with  sonic  communication,  Transmission  time  delay 
poses  prohlems  of  correspondence  between  display  and  control  (see 
Sections  5,2  and  5.3) . 

12.  Tracking,  i.e,,  keeping  the  video  or  sonar  sensor  positioned, 
oriented  and  focussed  on  a moving  target,  can  be  a difficult  and 
time  consuming  task  for  the  operator,  and  not  always  successful. 

Other  aspects  of  this  table  will  be  discussed  later  in  this  section. 

Sonic  Imaging 

An  alternative  to  video  imaging  is  sonic  imaging,  whereby  the  differ- 
ential sound  energy  impinging  on  each  point  in  a two-dimensional  array  of 
accoustic  transducers  or  a continuous  surface  of  piezo-electric  material 

< is  electronically  converted  to  a video-type  display.  Because  of  the  great 
reflecti ve-disr ipative  property  of  sound  in  water,  the  placement  arid 
frequency  of  two  or  more  sound  sources,  and  the  focussing  of  impinging 
sound  through  an  acoustic  lens  and  an  aperture  onto  the  transducer  are 
critical.  While  the  range  of  sonic  imaging  is  severely  restricted  (e.g., 
not  more  than  a dozen  feet  at  the  present),  the  fact  that  it  is  unaffect- 
ed by  water  turbidity  has  motivated  great  interest.  The  art  is  yet  primi- 
tive, however.  An  excellent  recent  review  is  by  Sutton  (1977). 

Optic-Sonic  Trade-offs 

< . . Whether  for  video  or  for  sonic  imaging,  the  trade-offs  between  range 

(distance  from  sensor  to  object),  energy  required,  turbidity  of  water  and 
resolution  obtained  are  of  major  interest  in  control  of  teleopera tors. 
Trade-offs  between  any  two  of  these  variables  depend  on  the  levels  of  the 
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OPTICAL  VS.  SONIC  IMAGING.  These  performance  surfaces  represent 
some  of  the  trade-offs  between  optical  and  sonic  sensing  in  water 
The  greatest  resolution  is  for  light  but  only  out  to  a limited 
range.  Acoustical  performance  does  not  degrade  as  turbidity 
increases  (left  graph),  but  does  degrade  as  less  energy  is  used 
for  insonification  (right  graph). 

Optical  Derformance  degrades  rapidly  with  turbidity  (left  graph) 
but  it  is  less  dependent  on  the  amount  of  power  used  than  is  the 
sonic  system  (right  graph). 
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other  variables  as  well  as  many  other  factors.  In  Figure  5.1  are  shown 
two  three-dimensional  plots  for  the  four  variables,  and  on  each  are 
plotted  trade-off  envelopes  for  both  video  and  sonic  Imaging,  indicating 
for  each  a set  of  approximately  equivalent  combinations  of  the  three  vari- 
ables on  that  plot  for  the  given  sensor  mode. 

Proximity  Sensing 

For  the  purposes  of  detecting  close  proximity  of  solid  objects  to 
a manipulator  hand,  to  control  a movement  along  but  not  touching  a surface, 
or  to  avoid  obstacles,  proximity  sensors  are  used.  These  can  be  based  on 
optical  (e.g.,  light  emitting  diode)  or  sonic  transducers,  in  either  case 
determiiiing  proximity  on  the  basis  of  reflected  energy. 

Force  Sensing 

Another  class  of  exteroceptors  (sensors  of  energy  imposed  from  outside 
the  teleoperator  system)  have  to  do  with  mechanical  forces  imposed  on  the 
teleoperator,  primarily  on  manipulator  arms  and  hands.  Sometimes  these  take 
the  form  of  wrist-force  sensors  (multi-degree  of  freedom  flexible  elements 
with  strain  gauges  attached  to  sense  displacement,  i.e.,  resolve  force  in 
each  translational  and  rotational  direction).  The  electrical  current  load 
on  the  drive  motors  of  the  arm  and  hand  give  the  same  information, 

after  subtracting  for  friction,  gravity  and  inertial  forces  and  resolving 
forces  through  the  complex  geometry  of  multiply  cantilevered  linkages  . 

"Force  sensors"  usually  convey  only  the  integrated  forces  imposed  on 
the  teleoperator,  In  contrast,  "tactile  sensors"  potentially  convey  the 
pattern  of  spatially  differential  forces  applied  to  the  teleoperator  from 
outside,  i.e.,  patterns  of  forces  on  the  "skin"  or-  surface  of  the  manipulator’s 
gripper.  Control  performance  in  the  use  of  some  of  these  sensors  is  discussed 
in  Section  7. 

There  are  many  other  sensors  which  perform  specialized  measurements  on 
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variables  such  as  depth,  speed  relative  to  ambient  water,  temperature,  acous- 
tic noise,  gravity,  salinity,  nutrients,  pollutants,  etc.  which  form  part  of 
the  teleoperator's  sensor  complement  when  needed. 

5 . 2 Communication,  its  Control  and  Environmental  Constraints 

When  a tether  is  used,  communication  between  teleoperator  and  human 
operator  is  normally  accomplished  by  electrical  means,  using  a coaxial  cable 
to  carry  video  signals  and  control  signals  multiplexed,  and  often  with  power 
sent  on  the  same  cable. 

Achievement  of  satisfactory  signal-to-noise  ratio  is  not  normally  a 
problem  in  such  cases.  The  problem  lies  more  with  the  mechanical  properties 
of  the  tether  - its  size,  weight,  drag,  and  problems  of  playing  it  out,  wind- 
ing it  in,  and  fouling.  The  use  of  kevlar  type  synthetics  to  provide  improved 
strength  to  weight  ratio  is  becoming  increasingly  prevalent. 

Optical  fibers  provide  the  capability  for  extremely  high  bandwidths 
which  add  a measure  of  redundancy  and  still  allow  high  bit-rate  two-way 
communication  between  a local  and  remote  computer.  Improvements  in  coatings 
to  reduce  refractive  radiation  losses  now  allow  use  of  light  fibers  to 
any  depth,  provided  a few  repeaters  are  used.  If  the  optical  tether,  plus 
a sheath  to  provide  tensile  strength  and  abrasion  resistance,  are  not  ac- 
companied by  a tether  to  carry  electrical  power  and/or  mechanically  pull  up 
the  submersible  vehicle  itself,  such  tethers  car  be  smaller  than  convention- 
al coaxial  cable  tethers  by  a factor  of  approximately  ten  in  diameter  and 
viscous  drag. 

When  there  is  no  electrical  or  optical  communication  channel,  either 
sonic  communi cations  or  direct  conduction  through  the  water  are  the  communi- 
cation options.  The  latter  is  too  new  to  evaluate.  The  former  has  been 
a standard  means  for  underseas  communi cation  for  some  time  at  relatively  low 
frequencies,  but  as  carrier  frequencies  (bandwidths)  get  higher  and  distances 
get  longer  the  fraction  of  energy  lost  becomes  so  large  as  to  be  unacceptable. 
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Figure  5.2  shows  the  bandwidth  normally  possible  for  alternative  under- 
sea communication  systems:  fiber  optics  (3M  bits  per  second  and  up),  coaxial 
cables  (300K)  and  acoustic  conmuni cation  (up  to  30K  depending  on  range).  For 
each  such  system  is  indicated  the  trade-off  between  bits  per  frame  (number  of 
video  lines  makes  it  more  graphical  - regular  broadcast  TV  is  about  500)  and 
frames  per  second. 

By  contrast  to  electromagnetic  propagation,  which  is  instantaneous 
on  a human  response  time  scale,  propagation  of  acoustic  signals  may  pose 
a significant  time  delay.  Sound  travels  approximately  1600  meters  per  second 
in  water.  Considering  round  trip  time  delay  this  means  that  for  each  800 
meters  of  depth  there  is  a minimum  of  a full  second  between  the  time  a message 
is  sent  from  the  surface  and  the  time  any  feedback  of  results  of  that  message 
is  returned  to  the  surface.  The  time  may  be  lengthened  for  any  control  signal 
by  serial  encoding  and  decoding  at  each  end,  including  sharing  (multiplexing) 
with  other  signals.  Thus  for  300  m depths,  time  delays  might  be  1/2  second; 
for  600  m depths,  delays  might  be  1 second,  and  so  on. 

The  reason  why  time  delays  in  control  loops  are  undesirable  is  that 
they  cause  instabilities.  Specifically,  for  that  frequency  for  which  the 
time  delay  plus  inherent  dynamic  lag  is  one-half  cycle,  or  any  higher  frequen- 
cy, if  the  loop  gain  exceeds  one  the  system  will  become  unstable.  Human 
operators  can  avoid  this  problem  by  moving  and  stopping  (thereby  reducing 
gain  to  zero),  but  this  makes  direct  telemanipulation  cumbersome.  Experiments 
by  Ferrell,  Black,  Hill,  Starr  and  others  have  clearly  indicated  that  delays 
of  1 second  may  increase  task  times  by  four  or  more  times.  (Time  delay 
experimental  performance  data  are  further  reviewed  in  Section  7.) 

5.3  Display 

Conventional  Picture  Display  Technology 

Referring  to  the  display  column  of  Table  5.1,  it  is  clear  that  display 
variables  also  make  a critical  difference  in  the  vision  of  remote  objects 
both  for  inspection  and  for  manipulation.  However,  the  human  factors  of  CRT 
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FIGURE  5.2  TV  BANDWIDTH  REQUIREMENTS.  Normal  broadcast  television  has  a 

frame  rate  of  30  frames/second  and  resolution  of  about  500  lines. 

A long  range  (5  km)  acoustic  communication  link  for  an  untethered 
vehicle  might  provide  a very  grainy  picture  (50  lines)  every  2 
seconds  or  more  resolution  at  greater  intervals  (e.g.,  100  lines 
every  15  seconds). 

Some  important  questions  are:  how  communication  restrictions  affect 
task  performance;  for  different  tasks  what  is  the  appropriate  trade- 
off between  frame-rate  and  resolution;  and  whether  the  operator 
should  be  provided  the  capability  to  adjust  the  trade-off  himself 
while  operating. 
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and  other  displays  are  not  unique  to  undersea  operations.  It  may  be  said 
that  the  undersea  application  of  new  sophisticated  visual  display  technolo- 
gy, computer  generated  graphic  and  alphanumeric  information,  superimposed 
on  video  displays  or  separately  presented,  has  probably  lagged  behind  cor- 
responding aerospace  application.  If  the  display  of  such  information  is 
properly  integrated,  it  can  make  navigation,  power  use,  efficiency  of  search, 
etc.  quicker  and  more  reliable. 

A specific  example  is  the  use  of  special  predictor  display  technology 
where  there  are  transmission  time  delays,  as  described  in  the  next  subsection. 

Predictor  Display  Aids  When  There  are  Transmission  Delays 

For  remote  control,  there  are  two  sources  of  difficulty  with  sonar 
communications:  time-delay  and  slow-frame-rate.  Round  trip  time-delay  is 
the  time  for  a command  to  travel  to  the  vehicle  and  the  first  indication  of 
response  to  travel  back.  At  a minimum  this  will  be  two  times  the  distance 
divided  by  the  speed  of  propagation,  ?1 . For  example,  T = 1 second  at  about 
5,000  feet.  Pictorial  information  from  television  camera  or  obstacle  avoid- 
ance sonar  will  be  further  delayed  because  of  limited  channel  capacity. 

Assuming  a low  resolution  picture  of  80  K bits  and  a channel  capacity  of  10  K 
bits  per  second,  there  would  be  at  most  one  picture  every  8 seconds  ( S = 8 
seconds).  The  effects  of  trying  to  navigate  with  just  this  pictorial  infor- 
mation are  illustrated  in  Figure  5.3. 

One  way  around  this  difficulty  is  a predictor  display.  Predictor  displays 
were  first  investigated  for  submarine  control  (Kelley,  1968).  NASA  con- 
sidered predictor  displays  for  remote  control  of  unmanned  lunar  roving 
vehicles,  (Arnold,  1963)  but  sent  men  instead.  A predictor  display  for 
untethered  vehicle  control  would  present  a symbol  superimposed  on  the  slow- 
frame-rate  and  time-delayed  picture  from  the  vehicle's  television  camera. 

The  symbol  responds  instantaneously  and  continuously  to  the  operator's  com- 
mands predicting  "future"  positions  of  the  vehicle.  For  example,  referring 
to  Figure  5.3,  when  picture  0 is  complete  the  predictor  symbol  would  show 
the  position  1'.  Before  the  next  picture  from  |T)  arrives,  the  symbol  will  be 
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FIGURE  5.3  DELAYS  IN  PICTORIAL  FEEDBACK  for  remote  vehicle  control  through  a 
sonic  link  are  due  to  travel  time  (T,  distance/speed  of  sound)  and 
scan  time  (S,  bits  per  frame/bits  per  second).  The  vehicle  may 
hove  moved  considerably  from  where  the  operator  thinks  it  is. 

The  picture  Q is  received  T + S seconds  after  it  is  taken,  the 
first  operator  response  is  received  by  the  vehicle  at  least  T seconds 
later,  for  a total  delay  of  2T  + S seconds.  While  the  operator  is 
looking  at  the  still  picture  from  0 the  commands  he  is  sending  are 
actually  moving  the  vehicle  from  JVJ  to  as  illustrated  in  the 
lower  map  of  vehicle  motion  and  fields  of  view. 
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moved,  in  response  to  the  operator's  commands,  to  position  2'.  The  posii ion 
of  the  vehicle  is  computed  from  a local  model  of  the  vehicle  responses  and 
from  the  operator's  commands  u(t),  as  shown  in  Figure  5.4. 

The  predictor  symbol  may  prove  useful  both  on  pictorial  displays  (super- 
imposed on  television  or  obstacle-avoidance  sonar)  and  on  map-like  position 
displays.  Map  displays  would  avoid  one  difficulty  of  pictorial  displays,  which 
is  losing  the  predictor  symbol  when  it  moves  out  of  the  field  of  view  of 
the  camera  (for  example,  moving  sideways  or  backward). 

If  position  data  is  available  from  transponders  or  locator  beacons,  it 
could  be  used  to  update  the  vehicle  model.  With  just  the  pictorial  data, 
the  open-loop  prediction  would  have  to  span  an  interval  of  (at  least)  2T  + S 
to  (at  most  ) 2T  + 2S  seconds.  With  auxiliary  feedback  the  open-loop  estimate 
will  only  need  to  span  the  delay  of  that  auxiliary  data  (at  minimum  2T).  The 
signals  and  corresponding  delays  are  shown  in  Figure  5.4.b.  (u(-),  command 

vector;  x(*)»  vehicle  location  data).  Another  feature  that  could  be  built 
into  the  local  model  of  the  vehicle  is  some  estimate  of  the  disturbances 
(such  as  current).  The  model  of  the  current  as  well  as  the  vehicle  model  could  be 
updated  on  the  basis  of  the  mismatch  between  predicted  and  measured  vehicle 
position. 

One  unexpected  finding  from  simulation  experiments  (Verplank,  1978) 
was  that  rather  than  sending  the  picture  periodically  every  eight  seconds 
sending  the  picture  only  upon  the  operator 's  request  reduces  the  total  number 
of  pictures  necessary  and  encourages  a "move  and  wait"  strategy  which  avoids 
confusion.  The  difference  is  illustrated  in  Figure  5.5. 

On  an  actual  vehicle,  probably  both  modes  should  be  available  with  the 
request  mode  used  when  move-and-wait  strategy  is  appropriate  (for  precise 
positioning  based  on  pictorial  feedback,  and  when  environmental  disturbances 
are  small).  Periodic  mode  is  probably  more  appropriate  for  less  precise 
navigation  and  continuous  motion  when  the  predictor  symbol  can  be  relied  upon. 
Another  trade-off  that  should  probably  be  built  into  the  pictorial  feedback 
is  variable  frame-rate/resolution.  In  a more  dynamic  and  uncertain  environ- 
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FIGURE  5.4  PREDICTOR  DISPLAYS  present  the  operator  with  dynamic  information  j I 

important  for  good  control  which  he  would  otherwise  have  difficulty  ; | 

estimating.  For  example,  a sonic  link  might  give  delayed  (time  T)  j 1 

and  slow-frame-rate  pictures  (5  seconds/frame)  at  least  2T  ■*  S , .a 

seconds  "old".  A local  computer  model  of  the  vehicle's  response  1 J 

is  used  to  calculate  and  display  the  "current"  estimated  position  ■ J 

of  the  vehicle,  5<(t)  on  the  basis  of  the  operator’s  commands  u(t)  ■ II 

and  possible  auxiliary  position  data  (x(t  - 2T),  dotted  line).  j | 

The  predictor  symbol  could  be  displayed  in  perspective  superposed  5 J 

on  the  delayed  picture.  (Verplank,  1 978) . | J 
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FIGURE  5.5  SLOW  FRAME-RATE  PICTURES  could  be  sent,  either  continually  (every 
5 seconds  in  a "PERIODIC  MODE",  or  only  upon  the  operator's 
command  "REQUEST  MODE",  It  has  been  discovered  that  there  can  be 
a large  effect  on  the  quality  of  control,  with  or  without  a predictor 
display  (Verplank,  1978), 

In  the  periodic  mode  a short  move  starting  with  the  receipt  of 
picture  gjwill  not  be  reflected  in  the  next  picture,  Q > as  the 
operator  might  expect;  instead  he  has  to  wait  for  . In  request 
mode,  the  wait  for  pictorial  confirmation  is  minimized. 


ment  (i.e,,  larger  bandwidth  disturbances  or  target  motion)  sampling  rate  will 
want  to  be  higher  at  the  expense  of  resolution. 

Display  of  Integrated  or  Differentiated  Forces 

The  most  natural  display  for  integrated  (resultant)  forces  imposed  on 
the  manipulator  is  the  bilateral  or  force-reflecting  master-slave  manipulator. 
Forces  can  be  scaled  to  be  some  constant  proportion  of  (usually  less  than) 
the  imposed  forces,  and  if  desired,  the  positional  differences  which  correspond 
to  the  force  differences  can  also  be  scaled  through  use  of  a smaller  size 
master  than  slave.  Display  of  wrist  forces  (6  degrees  of  freedom)  and  tactile 
forces  (spatial  distribution  at  various  points  on  the  manipulator  hand)  is 
more  difficult.  Experimenters  with  such  devices  usually  have  chosen  to 
display  such  information  visually  (though  there  is  difficulty  for  experi- 
mental subjects  to  make  the  cross-modality  transfer  from  what  would  normally 
be  muscle  senses  or  cutaneous  senses  to  vision).  Cutaneous  display  of  forces 
on  the  same  hand  as  is  controlling  a master-slave  manipulator  has  not  yet 
been  demonstrated  successfully;  supervisory  (computer)  control  of  manipulation 
may  make  it  practical  for  a monitoring  operator  to  assign  one  hand  to  feel 
the  forces  that  the  remote  manipulator  feels, 

5.4  Vehicle  Mobility,  Navigation,  and  Vehicle  Use  as  a Platform  for 

Manipulators  and  Sensors 

Dynamic  and  control  aspects  of  submersible  vehicles  in  general  are  be- 
yond the  scope  of  this  report.  However,  the  remote  positioning  of  video  or 
acoustic  sensors  and  the  remote  control  of  manipulators  are  a function  of 
vehicle  motion. 

Some  submersibles  are  made  specifically  for  inspection  only  - moving  a 
video  camera  to  proper  position  and  holding,  or  moving  continuously  to  scan 
a cable  or  pipe  or  structure.  Whether  the  vehicle  is  tethered  or  not,  in- 
spection poses  significant  problems  for  teleoperator  control.  This  is  so  for 
several  reasons.  One  is  that,  while  operating  where  there  is  plenty  of  light 
and  the  medium  is  completely  clear  one  need  only  use  a zoom  lens  to  bring 
the  view  "close-up".  When  the  medium  has  suspended  particles  which  reduce 
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visibility  one  needs  to  swim  up  close  for  a decent  view.  But  swimming  up 
close  may  be  hazardous  because  of  collision,  or  because  of  danger  of  foul- 
ing a tether.  A second  reason  is  that  ocean  currents  may  be  so  strong  as 
to  prevent  "hovering"  without  a large  expenditure  of  energy;  continually 
drifting  past  the  object  of  attention  may  be  a frustrating  way  to  do  visual 
inspection. 

When  the  teleoperator's  task  Is  to  manipulate,  as  well  is  to  observe, 
the  problems  may  be  more  severe.  To  perform  mechanical  work  on  an  object, 
i.e.,  apply  forces  with  manipulator,  there  must  be  some  mechanical  way  to 
resist  the  opposing  forces  which  necessarily  occur  on  both  manipulator  and 
environmental  object.  If  both  vehicle  and  environmental  object  are  resting 
firmly  on  the  bottom,  this  may  be  no  problem.  But  if  either  is  not 
resting  firmly,  i.e.,  is  free  to  rock  or  glide  or  is  being  pulled  by 
ambient  current  forces  on  both  vehicle  and  tether,  or  if  either  is  free- 
swimming,  it  is  necessary  to  employ  one  or  more'grabber  arms'  to  hold  the 
environmental  object  relative  to  the  manipulator.  These  can  be  simple 
manipulators,  with  fewer  degrees  of  freedom  and  less  dexterity  than  the 
work-manipulator.  But  one  must  first  find  a place  to  grab.  Further,  once 
having  grabbed, a pivot  point  is  established;  forces  applied  by  ambient  cur- 
rents or  by  the  manipulator  may  tend  to  rotate  the  submersible  and  task 
object  relative  to  one  another  around  the  pivot  point.  This  is  especially 
serious  if  the  submersible  or  the  task  object  is  large,  where  movement  is  slow 
and  unnoticed  in  the  acceleration  phase,  and  stopping  such  motion  abruptly 
to  stabilize  a critical  manipulative  activity  is  not  possible. 

Often  both  the  submersible  and  the  manipulator  are  controllable  in 
six  degrees  of  freedom.  This  means  there  is  redundancy,  and  possibly  con- 
trol of  the  vehicles'  degrees  of  freedom  can  be  used  to  replace  or  augment 
the  manipulator's  degrees  of  freedom.  Little  significant  work  has  been 
done  as  yet  on  this  topic. 

Manipulator  arms  and  associated  video  sensors  can  be  "packaged"  in  a 
structure  which  itself  can  be  easily  attached  to  and  detached  from  one  or  more 
submersible  vehicles.  This  permits  a variety  of  work  platforms,  depending  on 
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depth  and  other  conditions,  ano  does  not  burden  any  one  vehicle  with  having 
to  carry  extra  equipment  when  not  needed.  An  example  is  the  Navy's  "Work 
Systems  Package  ,a  description  of  which  is  quoted  below  (Bertsche  et  al.,  1978). 


"The  Work  Systems  Package  (WSP)  was  developed  and  fabri- 
cated under  the  Navy's  Deep  Ocean  Technology  Program  and 
is  designed  to  provide  a versatile  work  capability  to  depths 
of  20,000  ft.  The  WSP  is  a group  of  manipulator  arms  end 
tools  integrated  into  a modular  package  that  will  provide 
a heavy  duty  work  capability  when  mounted  as  a unit  on  the 
Navy's  CURV  III  or  RUWS  unmanned  cable  controlled  sub- 
mersible vehicles,  and  the  ALVIN,  SEACLIFF  and  TURTLE 
manned  vehicles.  In  addition,  it  can  be  positioned  and 
controlled  by  divers  or  operated  independently  from  a 
surface  support  ship  for  operations  at  shallow  depths  with- 
out the  necessity  of  resurfacing  for  tool  interchange. 
Potential  tasks  include  salvage,  recovery,  installation, 
and  repair  operations.  Basic  components  of  the  work  pack- 
age include  two  simple  outer  manipulator  arms  without 
elbow  functions  that  act  as  ’grabbers'  or  restraining/ 
holding  arms  to  steady  the  vehicle  or  hold  small  work  pieces. 

A centrally  located  seven-function  manipulator  arm  can 
select,  interchange,  and  operate  a variety  of  hydraulically- 
powered,  explosively-actuated,  or  electrically-actuated  tools, 
Included  in  the  tool  storage  box  are  tools  to  perform  cable 
cutting,  synthetic  line  cutting,  nut  torquing,  jacking, 
prying,  wire  brushing,  sawing,  grinding,  drilling,  chipping, 
and  stud  driving.  An  electrically-driven  hydraulic  pump  unit 
supplies  the  power  to  a majority  of  the  tools.  Electric  power 
is  supplied  from  a self-contained  battery  package.  Control  of 
all  operations  and  functions  is  provided  through  a multi- 
plexed telemetry  circuit  from  the  vehicle.  Pressure  insen- 
sitive electronic  circuits  and  pressure  compensated  hydraulic 
components  allow  all  systems  to  operate  at  full  ambient  pres- 
sure." 


5 . 5 Arms  and  Hands  Design  and  Contro 1 


While  our  purpose  in  this  report  is  not  to  be  concerned  with  mechan- 
ical arm  and  hand  design  per  se,  there  are  aspects  of  arm  and  hand  design 
which  are  important  to  man-machine  control. 
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Most  manipulator  arms  have  six  degrees  of  freedom  from  shoulder  to  wrist 
just  enough  to  place  the  wrist  at  any  position  and  orientation  within  a 
working  envelope.  The  larger  the  working  envelope  the  larger  the  manipula- 
tor and,  usually,  the  slower.  Some  manipulator  arms  have  only  four  or  five 
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degrees  of  freedom  between  shoulder  and  jaw,  allowing  the  end  effector 
to  make  up  for,  say,  wrist  rotation  by  having  its  own  wrist  rotation  or  drill- 
ing possibility,  or  allowing  the  angle  of  grasp  to  compensate  in  some  way 
for  one  degree  of  freedom. 

Figure  5.6  illustrates  two  popular  types  of  kinematic  design  - the  one 
consisting  entirely  of  rotary  joints,  the  other  including  one  prismatic 
joint.  Prismatic  joints  accommodate  to  hydraulic  control  because  linear 
piston-cylinder  actuator  combinations  are  simple;  however,  prismatic  bear- 
ings present  problems.  Rotary  joints  of  course,  can  be  actuated  by  linear 
piston-cylinders  on  linkages,  or  by  rotary  hydraulic  motors,  which  have 
seen  much  improvement  in  recent  years. 

The  distances  between  joints,  and  the  types  of  joints,  whethe*-  rotary 
or  prismatic,  directiy  determine  the  limiting  envelopes  or  "approach  angles" 
whicn  constrain  the  orientation  of  the  last  link  as  the  terminal  device 
approaches  a required  point.  The  necessary  configuration  of  the  other 
joints  - elbow,  etc.,  are  also  thus  determined  - which  is  important  in 
case  the  manipulator's  working  space  is  not  free  but  constrained  by  objects 
to  be  "worked  around".  The  kinematic  analysis  of  such  problems  is  presented 
formally  by  Roth,  Kobrinskii  and  others  (Sheridan,  1976), 

The  design  of  the  kinematic  "linkage  from  shoulder  to  wrist  is  predicat- 
ed not  only  on  kinematic  considerations  for  reaching  a given  point  and 
orientation  in  cartesian  space  and  avoiding  collision  of  intermediate  link- 
ages, but  also  on  considerations  of  statics  (beam  defection  analysis),  dynamics 
(oscillations  due  to  mass  of  the  arm,  viscoelasticity  of  solid  and  fluid  sus- 
pension) and  motive  power,  (e.g.,  maintaining  sufficient  static-force  to  hold 
up  a weight  against  gravity  while  in  outstretched  position;  also  the 
attainment  of  sufficient  slewing  speed). 

Design  of  the  hand,  or,  more  generally,  the  "terminal  device"  is  usually 
regarded  as  a separate  problem.  One  philosophy  is  to  have  a very  general 
purpose  hand  with  at  least  grasp  and  possibly  other  degrees  of  freedom  (e.g., 
bending  fingers).  The  alternative  philosophy  is  to  have  a whole  "tool-box" 
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FIGURE  5.6  MANIPULATOR  KINEMATICS  generally  fall  into  two  classes,  those 
with  elbows  (e.g.,  all  ROTARY  joints)  and  those  without  (e.g., 
one  PRISMATIC  joint).  There  are  open  questions  about  which 
configurations  are  best  for  computer-control  (Roth,  1976)  and  which 
may  he  best  for  human  control. 


of  power-driven  terminal  effectors  any  of  which  can  be  changed  for  another 
under  remote  control  in  the  process  of  doing  work.  The  Navy's  Work  Systems 
Package  has  a number  of  such  end  effectors,  which  can  be  stowed  in  the 
tool  box  by  applying  enough  force  {which  breaks  the  hydraulic  link  but  re- 
seals the  supply  tune  so  that  minimal  hydraulic  fluid  is  lost).  A differ- 
ent terminal  device  can  then  he  attached,  !.ydraulic  pressure  can  be  applied 
to  it,  and  it  can  be  removed  and  used.  Brushes,  saws,  drills,  taps,  torque 
wrenches  and  compact  hammers  are  popular  terminal  devices,  in  addition  to 
"grabbers" . 

Any  open-chain  or  series  kinematic  linkage  will  suffer  from  the  common 
mechanical  problems  of  droop  under  load  (due  to  its  own  weight  plus  thatof 
load),  static  friction,  hysteresis  (backlash)  and  vibration.  These  problems 
will  be  different  for  each  combination  of  arm  configuration  and  load;  and 
electromechanical  position  sensors  in  the  joint  articulations,  depending 
on  how  they  are  connected,  will  not  necessarily  indicate  the  extent  of 
these  problems.  These  factors  pose  significant  difficulties  for  maintaining 
end-point  accuracy  in  a manipulator  under  computer  control,  if  it  is 
without  some  means  to  detect  the  actual  position-orientatior.  of  the  end 
point.  When  a human  operator  is  continuously  observing  and  controlling 
such  a manipulator,  however,  the  correctio;  of  such  a lack  of  isomorphism 
between  the  "program"  and  the  master  end  and  the  actual  behavior  at  the 
slave  end  occurs  relatively  easily  < id  naturally. 

5.6  Comr  and  Hardware,  Analogic  and  Symbolic;  Teleproprioception  and 

Anthropomorphism 

By  command  hardware  we  mean  those  devices  by  means  of  which  the  human 
operator  communicates  his  will  to  the  computer  and/or  teleoperator.  Such 
communication  can  be  through  voice-activated  devices,  through  pedals,  through 
electromyographic  signals,  or  through  other  body  "pick-ups".  But  hand-control 
predominates . 

Hand  controls  can  be  either  of  two  types:  analogic  or  symbolic.  Ana- 
logic controls  are  those  for  whi,.h  the  operator's  hand  motion  is  in  some  way 
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physically  analogous  or  isomorphic  to  the  display  response  desired:  moving  * 

a joystick  left  to  command  leftward  motion  of  the  display  and  at  a rate  pro- 
portional to  the  magnitude  of  the  joystick  movement.  Master-slave  position 
controls,  either  full-size  or  replica,  are  analogic.  Knobs  and  sliders  are 
analogic.  Switches  which  are  mounted  so  that  the  direction  of  throw  cor- 
responds to  display  movement  direction  may  be  considered  analogic. 

Symbolic  controls,  by  contrast,  are  not  particularly  physically  isomor- 
phic to  the  events  commanded.  Depressing  a button  doas  not  mean  the  response 
observed  on  the  display  moves  down.  For  symbolic  coding  - what  keys  are 
depressed  in  which  order  and  in  what  combination  with  other  keys  is  what 
determines  the  displayed  response.  Similarly,  voice  coronands  are  symbolic; 
the  sounds  have  given  meanings  which,  as  sounds, are  not  physically  isomorphic 
with  those  meanings. 

An  important  and  time-honored  human-engineering  principle  is  called 
"stimulus-response  compatibility".  The  events  observed  on  the  display  should 
have  a "natural  correspondence"  in  time  and  space  with  human  responses 
made  or  called-for,  If  this  "natural  correspondence"  is  not  present  the 
human  operator  tends  to  get  confused  and  make  errors. 

Compatible  di recti on-of -movement  relationships  will  improve  the 
performance  of  any  man-machine  system  by  improving  the  following:  reaction 
time  or  decision  time,  the  correctness  of  initial  control  movements,  the 
speed  and  precision  of  control  adjustment,  and  learning  time.  These 
improvements  are  relatively  unimportant  if  the  operator  has  a simple 
-epetitive  task,  but  their  importance  increases  with  "e  following:  the 
complexity  of  the  task,  the  discontinuity  or  number  . irruptions  in  the 
control  sequence,  the  degree  of  stress  or  anxiety  expei lenced  by  the  opera- 
tor (Morgan,  et  al . , 1963),  All  of  these  considerations  seem  pertinent  to 
the  present  situation,  and  support  the  inclusion  of  compatible  analogic 
controls  in  the  command  hardware  for  computer-aided  manipulation. 

Achievement  of  "stimulus-response  compatabi 1 i ty"  is  not  accomplished 
automatically  hy  analogic  controls,  nor  is  it  impossible  to  accomplish 
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with  symbolic  controls.  If  valves  in  a chemical  plant  were  controlled 
analogically  by  knobs,  and  the  feedback  of  valve  openings  were  displayed 
on  a control  panel  directly  adjacent  to  or  in  spatial  isomorphism  to  the 
knob,  this  might  produce  satisfactory  stimulus-response  compatability. 

If  the  displays  were  layed  out  in  a matrix  with  rows  labelled  by  letters 
and  columns  by  numbers,  then  a keypad  with  letters  in  series  from  A to  2 
and  numbers  from  0 to  9 might  also  provide  stimulus-response  compatability 
even  though  two  or  more  keys  might  be  required  to  select  a single  valve  and 
the  key  positions  would  not  be  isomorphic  with  display  positions. 


The  situation  in  teleoperator  control  is  compounded  by  the  need  to 
control  many  degrees  of  freedom  simultaneously,  in  a process  which  is 
physically  remote  from  the  operator,  not  always  in  view  in  the  display, 
and  possibly  delayed  in  time.  And  sometimes  (as  with  direct  switch-actuat- 
ed rate  control)  as  the  teleoperator  moves  relative  to  the  video  camera,  the 
observed  response  direction  of  its  movement  (of,  say,  the  manipulator  arm) 
for  a given  direction  of  hand  control  movement  changes.  In  other  words, 
continuous  display-control  compatability  is  not  possible  with  such 
controls;  the  display  is  not  to  be  taken  as  indicating  what  to  do  next. 


For  these  reasons  the  operator  can  lose  track  of  the  present  true 
configuration  of  the  teleoperator  relative  to  where  it  should  be,  relative 
to  what  he  observes  on  the  display,  relative  to  what  control  actuations 
he  should  make,  and  relative  to  his  own  body. 


If  the  teleoperator  is  designed  to  be  anthropomorphic,  the  operator 
tends  to  identify  his  own  body  and  his  immediate  environment  with  the 
remote  vehicle  and  its  environment;  he  identifies  his  own  arm  and/or  the 
hand-control  or  master-arm  with  the  remote  arm  attached  to  the  vehicle; 
and  he  identifies  his  head  orientation  with  the  orientation  of  the  remote 
video  camera  (which  of  course  is  producing  the  video  display  he  is  looking 
at).  His  "bridge"  to  inferring  what  is  going  on  in  the  remote  environment 
is  what  he  sees  happening  in  the  display  (i.e.,  the  position  and  orientation 
of  displayed  remote  arm  and  nanipulated  remote  object)  and  thus  the  corres- 
pondence between  the  displaw  and  his  body  movements  is  critical.  This 
effort  to  maintain  awareness  of  the  teleoperator  and  its  environment  by 


relation  to  his  own  body  and  its  environment  we  call  "teleproprioception". 
Teleproprioception  is  believed  to  be  of  great  importance  for  control. 

Figure  5.7  represents,  by  means  of  vectors  to  indicate  the  positions 
and  orientations  of  the  various  important  components,  and  by  identities  rela- 
ting these  vectors  to  each  other,  the  isomorphisms  which  obtain  for  perfect 
correspondence  (ideal  teleproprioception).  The  more  of  these  identities 
which  hold,  or  the  more  accurately  they  hold,  the  better.  For  example, 

C - B = A - V means  that  the  control  movement  relative  to  the  operator's 
body  corresponds  to  remote  arm  movement  relative  to  the  vehicle.  Figure  5.7 
is  intended  only  to  suggest  the  beginnings  of  an  analysis  of  this  problem. 

The  effects  of  lack  of  isomorphism  need  to  be  studied  in  detail. 

It  has  been  shown  empirically  that  small  deviations  from  the  above 
isomorphisms  can  be  compensated  for  quite  easily  by  the  operator.  For  example, 
Vertut  (1976)  showed  that  if  the  master  end  of  a master-slave  manipula- 
tor is  rotated  30°  relative  to  the  slave  (and  all  other  correspondences 
left  undisturbed)  the  operator  could  compensate,  but  as  the  disparity  went 
beyond  45°  performance  deteriorated  badly. 

Various  experimenters  have  employed  head-mounted  CRT  displays  to 
which  the  remote  camera  is  servo-positioned;  this  ensures  that  the 
operator's  head  orientation  relative  to  his  body  corresponds  to  the  camera's 
orientation  relative  to  the  vehicle,  and  what  he  sees  in  the  display  is 
as  if  he  were  physically  present  at  the  camera  mount  on  the  remote  vehicle. 
Unfortunately  these  experiments  have  not  proven  particularly  successful 
due  both  to  the  bodily  encumbrance  of  head-mounted  displays  and  to  poor 
mechanical  tracking  of  remote  video  camera  to  head.  Experimenters  in  the 
USSR  (Yastrebov  and  Stefanov,  1978)  have  placed  the  operator  in  a cab  which 
pitches  and  rolls  in  relation  to  vehicle  pitch  and  roll  to  provide  vesti- 
bular cues  for  teleproprioception. 

In  continuous  man-in-the-loop  control  anthropomorphic  teleoperator 
design  is  obviously  important.  But  anthropomorphic  design  becomes  less 
important  as  1)  control  becomes  more  autonomous,  and  2)  control  becomes 
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A Arm  of 
manipulator 

0 Object 
manipulated 

S Sensor 
(camera) 

V Vehicle 


FIGURE  5.7 


C Control  lever 
or  master 

A'  Arm  of 

manipulator 

(displayed) 

O'  Object  manipulated 
(displayed) 

S'  Video 
picture 

H Head  of 
operator 

B Body  of 
operator 


ISOMORPHISMS  REQUIRED  FOR  TELEPROPRIOCEPTION-  Vectors 
represent  positions  and  orientations  of  vehicle,  camera, 
manipulator  and  manipulated  object  (all  at  remote  site) 
and  of  operator's  body,  head  and  arm,  and  display  of 
remote  manipulator  and  object  (all  at  local  site). 
Identities  below  indicate  correspondences  which  hold  for 
perfect  isomorphism. 

B - V 

S'-  B = S - V,  5 = H 
C - B = A - V 
O'-  B = 0 - V 
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more  predictable.  If  the  dimension  of  anthropomorphism  is  added  qualitative- 
ly to  the  two  dimensions,  task  entropy  and  control  autonomy,  by  which 
teleoperators  were  classified  in  Figure  4.3,  Figure  5.8  results.  The 
hypothetical  rectangular  solid  of  all  teleoperators  is  truncated  on 
the  lower  left  (sliced  front-to-back)  because  ethically  we  seek  to  avoid 
assignment  of  people  to  such  low  level  undignified  tasks.  It  is  truncated 
by  the  upper  right  front-to-back  slice  because  we  aren’t  yet  clever  enough 
to  build  such  systems,  and  it  is  truncated  by  a right  rear  inward-slanting 
top-to-bottom  slice  because  of  the  abovementioned  absence  of  need  for  anthro- 
pomorphism. That  is,  automatic  teleoperation  could  just  as  easily  be  ac- 
complished by  non-anthropomorphic  design. 
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FIGURE  5.8  DOMAIN  OF  MAN-TOOL  SYSTEMS  IN  WHICH  ANTHROPOMORPHIC  TOOL  DESIGN 
IS  USEFUL.  This  is  an  extension  of  Figure  4.3.  Useful  man-tool 
systems  can  be  represented  by  a rectangular  solid-truncated  at 
the  lower  left  corner  by  our  refusal  to  employ  people  as  slaves, 
at  the  upper  right  corner  by  our  limits  on  knowledge.  It  is  also 
truncated  by  a right-rear  diagonal  slice  which,  for  good  design, 
requires  less  anthropomorphism  as  control  autonomy  increases  op 
as  the  task  entropy  decreases  (task  becomes  more  predictable). 
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Previous  sections  have  discussed  in  general  the  trade-offs  between 
human-  and  computer-control  of  teleoperation  (Section  4)  and  the  hardware 
necessary  (Section  5).  This  section  looks  in  detail  at  the  requirements 
for  a supervisory  control  language  and  describes  several  examples  of  existing 
programs.  The  emphasis  is  on  human  interaction  with  such  software.  We  pro- 
pose both  a general  framework  and  some  specific  principles  of  good  user- 
oriented  language  design. 

6-1  Sharing  vs.  Trading,  Transparent  vs.  Apparent 

One  useful  distinction  between  different  application  of  computers 
is  in  how  they  are  supervised  by  the  human  operator;  the  operator  either 
shares  or  trades  control  with  the  computer.  Here,  to  share  control  means 
that  both  human  and  computer  are  active  at  the  same  time.  To  trade 
control  means  that  at  one  time  the  computer  is  active,  at  another  the 
human  is. 

For  example,  the  servomechanism  of  a master-slave  system  is  a form 
of  automation  with  which  the  operator  shares  control.  Similarly,  real- 
time coordinate  transformations  (such  as  resolved-motion  rate-control)  are 
an  application  of  computer-aiding  where  the  operator  shares  control.  Other 
computer-aids  which  would  be  in  simultaneous  operation  are  sensor-condi- 
tioning, communication-channels  and  display-aids;  they  share  control  with 
the  operator. 

The  operator  trades  control,  for  example,  when  switching  from 
training  to  automatic  execution  of  a recorded  path.  Emergency  over- 
rides either  by  the  operator  or  by  the  computer  are  instances  of  trading 
control . 


Any  particular  implementation  of  computer-aiding  may  involve  both 
sharing  and  trading  control.  When  a pilot  changes  the  heading  on  an  auto- 
pilot, he  is  sharing  control  with  the  guidance  computer.  When  he  resumes 
manual  control,  he  is  trading. 

A proposed  design  principle.  This  trading/sharing  distinction  will 
be  used  in  the  next  section  to  classify  various  computer-aids  to  manipula- 
tion. The  distinction  may  have  an  important  bearing  on  how  the  man- 
machine  interface  should  be  designed.  The  demands  on  the  human  operator 
will  probably  be  quite  different  in  sharing  and  trading. 

It  is  proposed  here  that  in  sharing  the  computer  control  should  be 
" transparent" ; the  operator  should  be  in  continuous,  direct  control  with 
his  work  made  easier  by  computer  aiding.  "Transparent"  here  implies 
that  the  operator  "sees  through"  the  computer  control  directly  to  the 
task  to  be  done.  For  example,  a good  master-slave  system  with  force- 
feedback  allows  the  operator  to  identify  with  the  remote  arms  and  hands 
as  though  they  were  his  own. 

The  demands  on  the  operator  when  trading  control  have  to  do  with 
knowing  what  the  automatic  control  will  do,  is  doing  and  has  done  (that  is, 
he  must  plan,  monitor  and  intervene).  In  this  case,  the  comnuter  should 
be  "apparent" , not  transparent.  That  is,  in  planning  as  well  as  programming 
or  training  for  autonomous  operation,  it  should  be  apparent  to  the  operator 
v/hat  the  teleoperator  will  do  (and  is  capable  of  doing).  In  monitoring, 
the  teleoperator  and  environmental  states  should  be  apparent  to  the  operator, 
especially  if  he  needs  to  intervene. 

There  may  be  some  difficulty  with  this  notion  of  "transparent"  and 
"apparent"  computer-aiding.  Are  there  times  when  shared  control  should 
become  "apparent"?  Yes,  but  probably  only  when  control  is  being  "traded". 

For  example,  resolved-motion  rate-control  "shares"  control;  it  should 
normally  be  "transparent",  that  is,  the  operator  should  feel  that  he  i$  in 
direct  control  of  the  end-effector,  not  worrying  about  what  the  individual 
joint  velocities  are.  However,  if  the  computer  "fails"  for  example  by 


reaching  joint  limits  or  gimbal-lock,  this  should  be  "apparent"  to  the 
operator  so  that  he  can  "trade"  control  with  the  computer  and  command 
individual  joints  directly,  if  necessary.  If  there  is  no  need  to  trade 
control,  if  the  computer  can  handle  the  problem,  it  need  never  become 
apparent  to  the  operator. 

There  might  be  situations  where  transparency  of  shared  control  is  not 
appropriate  but  our  first  guess  is  that  this  is  a good  general  design 
principle.  Further  research  and  field  experience  may  show  whether  or  not 
this  is  so. 

6.2  Examples  of  Shared  Control 

Several  computer-aids  to  manipulation  where  control  is  shared  have 
been  demonstrated  in  the  laboratory.  In  general,  their  intent  has  been 
to  put  the  operator  in  more  direct  control  of  the  task  at  hand  without 
having  to  worry  about  one  or  more  of  the  difficulties  in  conmuni cation  or 
control;  they  are  successful  to  the  extent  they  are  unobtrusive  or  "trans- 
parent". 

Resolved  Motion  Rate  Control  is  a scheme  invented  by  Whitney  (1969) 
and  evaluated  by  Mullen  (1973),  where  the  computer  does  a real-time  coor- 
dinate conversion  between  the  operator's  commands  (rates  in  room-  or  hand- 
coordinates)  and  the  joint  rates  of  the  manipulator.  This  allows  the 
operator  for  example,  to  command  a linear  extension  or  sweep  of  the 
end  effector,  without  having  to  coordinate  which  arm  joints  to  move  at 
what  velocities. 

In  general,  this  kind  of  computer-aided  coordinate  conversion  would 
allow  the  operator  to  specify  the  desired  end-point  position  or  velocity  and 
have  the  computer  d>L(de  what  combinations  of  component  joint  articulations 
will  produce  that  desired  result. 
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Auto- Indexing  was  suggested  by  D.  Jelatis  (1977)  and  implemented  by 
Brooks  (1978).  If  the  range  of  the  master  in  a master-slave  manipulator 
cannot  be  as  large  as  that  of  the  slave  (such  as  on  a manned  submersible) 
one  possibility  is  to  have  a simple  difference  in  scale.  However,  this  makes 
small  motions  difficult.  Another  option  is  to  allow  a 1:1  correspondence 
but  only  within  a small  volume  of  the  master's  motion;  if  the  operator 
tries  to  push  the  master  outside  that  volume,  the  slave  is  "indexed"  at  a 
rate  proportional  to  how  hard  the  operator  is  pushing  and  in  the  appropriate 
direction.  The  result  is  an  offset  between  master  and  slave  positions,  but 
continued  master-slave  control  with  force-feedback.  If  the  allowed  volume 
of  master  motion  is  set  to  zero  this  becomes  a mixed  mode  of  control  where 
translations  of  the  end-effector  are  under  rate  control,  and  orientations 
are  under  position  control. 

Predictor  Displays  are  a form  of  computer-aiding  that  make  control 
easier  by  presenting  dynamic  information  to  the  operator  which  would 
otherwise  be  hard  for  the  operator  to  estimate.  They  have  been  proposed 
as  computer- a ids  for  better  human  control  of  remote  untethered  submersibles 
(see  Section  5.3  and  7.6).  In  a sense,  the  computer-aid  is  transparent  to 
the  operator  who  directly  controls  the  position  of  the  predictor  symbol; 
the  vehicle  then  "follows"  the  predictor. 

Automatic  Slaving  of  T.V.  Camera  to  Arm  Position  has  been  suggested  by 
Wernli  (1978)  for  application  on  the  Navy's  Remote  Underwater  Work  System. 
Their  studies  of  time  spent  in  camera  positioning  (for  several  salvage  scen- 
arios) show  nearly  10%  of  the  time  is  spent  on  camera  positioning.  Under 
computer  control  the  camera  could  be  positioned  to  always  keep  the 
manipulator  hand  in  view,  thus  alleviating  the  operator  of  the  necessity 
to  move  or  adjust  the  cameras. 

Active  Accomodation.  In  attaching  one  part  to  a mating  part,  if 
alignment  is  not  near  perfect,  forces  may  build  up,  the  objects  bind,  and 
the  desired  final  attachment  may  never  succeed.  This  can  be  avoided  with 
sufficiently  sensitive  force-feedback  on  master-slave  manipulators;  yet. 
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force-feedback  to  the  operator  may  be  undesirable  because  of  communication 
restrictions  (e.g.,  time-delay)  or  expense.  If  the  manipulator  is  equipped 
with  a suitable  wrist-force  strain  sensor,  or  with  tactile  sensors,  the 
computer  may  be  programmed  to  make  the  manipulator  "seek"  a least-force 
ith  which  moves  the  grasped  part  in  the  desired  direction  (Groome,  1972). 

This  is  a form  of  computer-aid  that  the  operator  might  share  control 
with;  the  operator  controls  gross  motions,  the  computer  makes  fine  adjust- 
ments upon  contact.  Accomodation  will  also  be  of  value  to  pre-programmed 
operations  where  the  operator  has  traded  control  with  the  computer.  In 
either  case,  the  properly  chosen  accomodation  scheme  could  be  "transparent" 
to  the  operator  or  the  program  cormianding  the  assembly  sequence. 

It  should  be  noted  that  passive  compliance  (the  mechanical  and  ser 
springiness)  is  vital  in  any  form  of  manipulation.  Some  things  would  bu 
impossible  without  it,  for  example  maintaining  contact  with  fixed  objects. 
For  specific  tasks,  special  fixtures  can  be  designed  which  have  the  appro- 
priate compliance  built  in.  For  example,  an  ingeneous  recent  development 
called  "remote  centered  compliance"  allows  a grasped  but  misaligned 
"peg"  to  translate  and  rotate  about  an  axis  displaced  from  the  manipulator* 
hand,  and  thus  align  itself  with  the  "hole"  while  still  being  forced  into 
mating  (Drake,  1977;  Whitney,  1978). 

6.3  Examples  of  Traded  Control:  Manipulation  Languages 

When  the  operator  trades  control  of  the  teleoperator  to  a computer, 
his  instructions  or  sequence  of  commands  to  the  computer  could  be  called 
a language.  For  manipulation  there  have  been  developed  two  kinds  of  language: 
world-modelling  and  explicit  (Park,  1977).  World-modelling  languages  are 
high-level  problem-solving  experiments  in  artificial  intelligence.  They  would 
accept  commands  such  as  "assemble  the  water  pump".  Explicit  manipulation 
languages  allow  the  operator  (or  the  world-modelling  program)  to  assemble 
sequences  of  primitive  manipulator  actions.  They  accept  conmands  such  as 
"CLOSE"  (the  jaw). 
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S ' World -Modelling  and  Artificial  Intelligence.  It  is  possible  to 

hypothesize  an  intelligent  teleoperator  with  enough  autonomy  to  be  called 
> ! a robot.  Many  of  the  components  are  under  active  development,  in  fact  much 

i j of  the  artificial  intelligence  research  has  used  manipulation  as  an  example 

j problem  (Winston,  19/7).  Natural  language  understanding  by  computer  was 

I j demonstrated  by  Winograd  for  the  restricted  case  of  arranging  blocks,  pyra- 

' mids  end  boxes  (the  "Blocks  World").  Scene  analysis  from  the  Blocks  World 

I was  solved  by  Waltz  by  exploiting  the  constraints  imposed  by  real  polyhedra 

on  their  1 i lie  drawings.  Some  of  the  earliest  work  on  problem  solving  was 
demonstrated  at  SRI  with  a general  problem  solver  (GPS)  called  STRIPS. 

i* 

t > 

- More  recently,  the  focus  of  problem  solving  research  and  the  planning 

i ; of  manipulation  has  shifted  to  the  representation  of  knowledge  for  parti- 

j(  ' 

, , cular  problem  domains.  Two  alternative  kinds  of  knowledge  are  identifiable: 

f j 1)  knowledge  about  the  actions  possible  in  that  domain  (PROCEDURAL)  and 

; 2)  knowledge  about  the  state  of  the  domain  (DECLARATIVE).  A currently  popu- 

; lar  structure  which  combines  these  is  Minsky's  Fr«.me  Theory. 

r.  ' 

' 

I , ' A successful  embodiment  of  the  Frame  Theory  is  NOAH  (Nets  of  Action 

S Hierarchies),  an  integrated  problem  solving  and  execution  monitoring  system 

t 

i intended  to  serve  within  a larger  computer  system  called  the  Computer-Based 

r 

; Consultant  (Sacerdoti,  1977).  NOAH's  knowledge  of  the  actions  in  its 

I world  is  encoded  in  a "procedural  network".  Two  phases  of  computer  planning 

j 

i are  used:  first  automatic  expansion  of  general  procedures  to  successively 

^ more  detailed  levels;  then  an  over-all  look  to  ensure  that  the  local 

i expansion  makes  global  sense. 

| Much  of  this  work  in  artificial  intelligence  has  been  aimed  at  urider- 

t standing  the  general  problems  of  intelligence, not  at  accomplishing 

i practical  manipulation.  For  example,  NOAH  has  planned  the  assembly  of  a water 

t 

pump  but  assumed  a human  as  a manipulator.  Other  current  world-modelling 
. languages  which  are  specifically  designed  for  the  planning  of  manipulation 

f are  AL  at  Stanford  University  (Goldman,  1977),  LAMA  at  M.l.T.  (Lozano-Perez, 

; 1977),  and  AUTOPASS  at  I.B.M.  These  are  all  sophisticated  programs  running 

[ on  large  computers.  Their  first  practical  contribution  will  probably  be 

[ in  planning  for  industrial  automation  of  mechanical  assembly.  They  are  a 
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long  way  from  being  on-board  controllers  for  teleoperators  In  uncertain 
environments. 

Other  planning  programs.  Several  other  experimental  computer  aids 
to  manipulation  fit  into  the  class  of  world-modelling  programs  but  at  a 
somewhat  less  ambitious  level  than  artificial  intelligence. 

Whitney  (1969)  showed  how  to  represent  manipulation  problems  with  a 
state-space.  Solutions  are  then  found  with  search  techniques  in  this  state- 
space.  He  found  that  for  simple  tasks  with  few  objects,  optimal  paths  are 
easy  to  find,  but  fhat  for  realistic  six  degree-of-freedom,  dynamic  objects 
and  manipulators,  the  options  are  simply  too  numerous  for  an  exhaustive  search 
in  a realistic  time. 

Hardin  (1970)  proposed  some  planning  heuristics  at  a somewhat  higher 
level  where  tasks  were  organized  in  an  AND-Tree  which  would  be  expanded  to 
detail  actions  and  trimmed  by  a critic  which  avoids  duplications  and  checks 
for  loops.  (This  is  very  similar  to  Sacerdoti's  procedural  nets.) 

Freedy  (1971)  and  Albus  (1973)  have  both  designed  learning  systems 
which  observe  the  human  operator's  control  of  the  arm  and  look  for  patterns 
of  motion  which  can  be  automatically  executed.  The  explicit  conmand  of 
arm  motion  appears  to  be  a more  practical  alternative. 

Explicit  Arm  Languages.  Today’s  general  purpose  industrial  robots 
have  very  simple  point-to-point  programs  which  are  usually  prograimed  by 
moving  the  manipulator  under  manual  control  to  successive  points  and  record- 
ing them.  This  record/ playback  mode  will  probably  be  one  of  the  first 
useful  computer-control  modes  for  undersea  teleoperators . Some  useful 
parallels  in  programming  details  might  be  worth  investigating.  We  give  no 
further  detai Is  here. 

Probably  the  first  computer  controlled  manipulator  which  could  respond 
purposefully  to  its  environment  was  MH1  programmed  by  Ernst  (1961).  His 
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BBP-1  program  could  be  used  by  the  human  operator  sitting  at  a keyboard,  for 
instance  to  build  small  structures  or  to  search  for  and  collect  blocks, 
by  calling  on  a range  of  simple  and  complex  subroutines.  The  manipulator 
jaw  was  equipped  with  a variety  of  touch  sensors  and  a simple  optical  proxi- 
mity sensor. 

A rather  complete  review  of  computer-controlled  manipulation  has  been 
compiled  by  Bejczy  (1973)  so  no  further  history  will  be  given  here.  What 
is  of  concern  are  the  details  of  user  interaction  with  such  computer-controlled 
manipulators.  Some  further  examples  will  be  given. 

The  C.S.  Draper  Laboratory  (1977)  has  an  arm  programming  language  which 
allows  the  programmer  to  write  a sequence  of  moves  and  tool  actions.  The 
computer  then  walks  the  arm  through  the  sequence.pausing  when  an  unspeci- 
fied arm  position  is  encountered. and  waits  for  the  programmer  to  move  the 
arm  there  manually  and  push  a "record"  button.  A program  sample  is  shown 
here: 


MC  EF  COLLETPICKUP  AS  M0VE(1 ,UC, 2,3,3, CG) 
MOVE ( 1 ,LC,7) 

MCDEF  COLLETINSERT  AS  M0VE(4,UC,5,6,6,CR) 
MOVE(4,LC,10) 

ATTACH  TOOL 
LOCK 

COLLET  RELEASE 
[NUT]  COLLETPICKUP 
COLLETINSERT 

[LOCKWASHER]  COLLETPICKUP 
COLLETINSERT 

[FANSPACER]  COLLETPICKUP 
COLLETINSERT 

[BEARINGSPACER]  COLLETPICKUP 
COLLETINSERT 
CLEAR  CR 
CLEAR  LC 
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The  arm  velocity  goes  to  zero  only  at  the  beginning  and  end  of  a MOVE 
sequence.  1,2,3  label  arm  positions  to  be  later  specified.  UC,  CG, 

LC  are  tool  motions  (unlock  compliance,  colletgrip,  lock  compliance),  which 
can  occur  while  the  arm  is  moving.  MCDEF  defines  a "macro*1.  Positions 
can  be  defined  locally  (labels  1-19)  within  a move.or  globally  (20-49). 

The  CSDL  system  of  teaching  by  showing  followed  by  record  playback  is 
similar  to  conventional  industrial  robots.  A significant  difference  is 
that  the  program  sequence  can  be  edited  without  losing  any  taught  data  and 
positions  which  must  be  returned  to  several  times  within  a program  can  be 
taught  once  and  referenced  by  number. 

SRI  International  is  developing  a slightly  more  sophisticated  user 
language  for  control  of  industrial  robots  (Rosen,  1977).  It  is  a FORTRAN- 
like  language  with  libraries  of  user-written  subroutines.  They  currently 
have  over  160  such  routines.  An  example  of  a user-language  application 
program  is  given  here: 


L$L00P 


COMMON 

DATA  NT I ME S= 10 
INTEGER  JUNK, 

P1=2,P2=4 

END 

PROGRAM  P$SAMPLE 
TYPEL  "SAMPLE  PROGRAM" 

ISETZ  COUNT 
INIARM 
JOYON  0 
INCHRV  JUNK 
TCOPY  (PI ,-#26) 

INCHRV  JUNK 
TCOPY  (P2,-#26) 

SSWAVE 

I PLUS  ( COUNT,  COUNT,  1) 

IFLSS  ( COUNT,  NTIMES ,L$LQ0P ) 


! RECEIVES  A TYPED-IN  CHARACTER 
! NUMBERS  OF  POSITION  TRANSFORMS 


! MAIN  PROGRAM 
! PRINT  THE  TITLE 

! START  UP  ARM 
! TURN  ON  JOYSTICKS 
! WAIT  FOR  A TYPE-IN 
! RECORD  1st  POSITION 
! WAIT  AGAIN 

! RECORD  SECOND  POSITION 
! CALL  SUBROUTINE  'WAVE* 
! DO  IT  "NTIMES" 

! IF  (COUNT.LT. NTIMES) 

GO  TO  L$L00P 


(continued  on  next  page) 
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! THTS  SUBROUTINE  WAVES  THE  ARM  BACK  AND  FORTH  BETWEEN 
! POSITIONS  PI  and  P2  . . . 


SUBROUTINE  S$WAVE 

MOVETO  PI 

WAITTC 

MOVETO  P2 

WAITTC 

RETURN 

END 


! START  ARM  MOVING  TO  PI 
! WAIT  TIL  IT  GETS  THERE 
! THEN  MOVE  IT  TO  P2 
! WAIT  TIL  IT  GETS  THERE 
! RETURN  TO  CALLER 


INCHRV  is  used  to  wait  for  the  operator  to  move  the  arm  manually 
and  then  type  in  any  character.  Then  TCOPY  records  the  current  arm  posi- 
tion. Note  the  limited  arithmetic  capabilities  (ISET7.,  IPLUS)  and  the 
ability  to  test  and  branch  (IFLSS). 

Rosen  e_t  aJL  (1977)  have  identified  user  interaction  as  an  important 
area  where  improvement  can  be  made.  Some  improvements  planned  are:  hot 
editing  (where  a program  can  be  modified  while  the  arm  is  executing  it), 
immediate  execution  (where  the  arm  executes  a statement  as  soon  as  it  is 
typed)  and  interactive  execution  (such  as  inserting  temporary  break-points 
where  control  is  transferred  to  the  operator). 

VAL  is  probably  the  most  sophisticated  explicit  prograrmii ng  language 
available  commercially  (Unimation,  1977).  It  allows  positions  to  be 
trained  by  manually  moving  the  manipulator  or  specified  in  terms  of 
room  coordinates  or  joint  coordinates,  or  hand  coordinates.  It  operates  in 
real-time,  simultaneously  issuing  manipulator  commands  and  interacting  with 
a human  operator  or  other  interactive  controller.  This  permits  on-line 
program  generation  and  modification  ("immediate  execution"  and  "hot 
editing" ) . 

MANTRAN  (Barber,  1967)  was  an  early  language  designed  especially  for 
interactive  supervisory  control  of  a teleoperator.  The  manipulator  was  equipped 
with  FRONT,  LEFT  and  BOTTOM  touch  sensors  and  the  language  allowed  branch- 
ing on  the  basis  of  this  information.  For  example,  here  is  a program  that 
moves  the  arm  down,  forward  and  left  until  it  touches  the  bottom-front-left 
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corner  of  its  work  space.  If  it  moves  2000  steps  before  touching  it 
stops  and  types" help". 


MOVE  TO  CORNER 


ST, 1 . MOVE  LEFT  2000  DOWN  2000  FORWARD  2000. 
UNTIL  A.)  TOUCH  LEFT 

B. )  TOUCH  FRONT 

C. )  TOUCH  BOTTOM 

IF  MOVE  CONDITION  SATISFIED,  HELP 
IF  A.)  DO  2 
IF  B.)  DO  5 
IF  C.)  DO  7 

ST.  2. MOVE  DOWN  2000  FORWARD  2000. 

UNTIL  A.)  TOUCH  FRONT 
B.)  TOUCH  BOTTOM 
IF  MOVE  CONDITION  SATISFIED,  HELP 
IF  A.)  DO  3 
IF  B. ) DO  4 

ST.  3.  MOVE  DOWN  2000. 

UNTIL  A.)  TOUCH  BOTTOM 
IF  MOVE  CONDITION  SATIFIED,  HELP 
IF  A.)  DO  8 


ST.  4.  MOVE  FOWARO  2000. 

UNTIL  A.)  TOUCH  FRONT 
IF  MOVE  CONDITION  SATISFIED,  HELP 
IF  A.  ) DO  8 

ST AT.  5.  MOVE  LEFT  2000  DOWN  2000. 
UNTIL  A.)  TOUCH  LEFT 
B.)  TOUCH  BOTTOM 
IF  MOVE  CONDITION  SATISFIED,  HELP 
IF  A.)  DO  3 
IF  B.)  DO  6 


i 

i 


ST AT.  6.  MOVE  LEFT  2000. 

UNTIL  A.)  TOUCH  LEFT 
IF  MOVE  CONDITION  SATISFIED,  HELP 
IF  A.)  DO  8 


STAT.  7.  MOVE  FORWARD  2000  LEFT  2000. 
UNTIL  A.)  TOUCH  FRONT 
B. ) TOUCH  LEFT 

IF  MOVE  CONDITION  SATISFIED,  HELP 
IF  A.)  DO  6 
IF  b.)  DO  4 


STAT.  8.  OPEN  1000* 


SI  AT.  9. 
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GO  TO  END 


There  were  several  limitations  which  MANTRAN  revealed.  There 
was  no  analogic  training  mode,  all  the  commands  had  to  be  typed  in  at 
a keyboard.  This  made  direction  and  position  specification  difficult  and 
"unnatural".  Another  difficulty  had  to  do  with  keeping  track  of  the 
complex  structure  of  the  program.  The  sample  above  is  a good  example  of 
how,  through  awkward  syntax,  the  multiple  branching  makes  the  program 
difficult  to  read,  understand  and  debug. 

Other  languages  for  supervisory  control  of  teleoperators  have  been 
written  at  SRI  (0.  Hill,  1973),  at  J.P.L.  (Bejczy,  1976)  and  at  Percep- 
tronics,  Inc.  (Shaket,  1977).  They  have  all  featured  the  ability  either 
to  control  the  manipulator  directly  with  some  form  of  manual  control  or  to 
have  the  computer  execute  a sequence  of  manipulator  conmands,  and  transfer 
control  back  to  the  operator.  Some  of  the  results  are  discussed  in 
Section  7.  A program  recently  developed  at  M.I.T.  is  describee  in  Section 
6.6. 


6 . 4 Elements  of  a Manipulation  Language 

In  an  attempt  to  organize  these  different  programming  languages, 
common  basic  elements  can  be  identified. 

ACTIONS.  Most  basic  are  the  primitive  manipulator  motions,  usually 
position  commands  or  velocities,  either  specified  absolutely  or  as  incre- 
ments either  in  joint  coordinates  or  some  more  convenient  frame. 

SUBROUTINES/PROCEDURES.  The  next  most  basic  programming  feature 
is  to  invoke  sequences  of  actions  by  name  or  number.  This  is  the  most 
powerful  simplifying  feature  to  the  human  operator.  Libraries  of  manipula- 
tion routines  allow  complex  operations  to  be  easily  and  quickly  programmed 
by  combining  pre- defined  sub-tasks. 

FLOW-CONTROL  is  usually  accomplished  with  tests  and  branching  to 
labelled  statements.  The  particular  form  of  conditionals  used  determines 
the  structure  of  the  program.  For  example,  GOTO  has  been  recently  con- 
demned because  it  leads  to  complicated  program  structures  difficult  to  under- 
stand and  debug  (Alagic,  1978),  The  tests  might  be  on  the  status  of  sensors 
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(e.g.,  touch)  or  on  variables  internal  to  the  program  (e.g.,  counters). 


1 

i 


VARIABLES/OPERATIONS.  The  simplest  variable  is  a named  position  of 
the  arm  which  can  be  returned  to.  Arm  motions,  or  sensor  data,  when 
properly  represented,  can  be  operated  on  in  various  coordi nate-framas . 

Most  languages  also  require  some  form  of  declaration  and  allow  assign- 
ment of  values  to  variables. 

INTERACTION.  Finally  there  must  be  some  form  of  interaction  with 
the  operator  or  programmer.  This  will  be  especially  important  to  teleopera- 
tors where  the  human  supervisor  will  be  available  as  monitor,  ready  to 
intervene.  Particularly  valuable  forms  of  interaction  are  teaching  of 
positions  through  a convenient  form  of  manual  control,  on-line  editing  of 
programs  (immediate  execution  and  hot  editing)  and,  of  course,  convenient 
means  for  trading  direct  control  of  the  manipulator  smoothly  from  human 
to  computer  and  back. 

6.5  Principles  of  Interaction 

It  seems  appropriate  to  discuss  several  design  trade-offs  which  raise 
questions  of  human  preference.  It  may  eventually  be  possible  to  formulate 
some  human  factors  principles  for  supervisory  control. 

Natural  vs.  Constrained  Language.  Ferrell  (1973)  has  presented  this 
issue  well  with  a simple  expe^'ment.  The  question  is,  how  constrained 
should  the  language  be?  Should  the  computer  only  respond  to  a small  set 
of  commands  or  should  it  allow  use  of  unconstrained,  natural  language? 

The  point  that  Ferrell  makes  is  that  for  simple  constrained  tasks  natural 
language  is  inefficient;  with  a special  purpose  vocabulary  suited  to  the 
task  the  human  supervisor  is  quicker  at  generating  sufficient  instructions. 
That  is,  independent  of  how  powerful  or  autonomous  the  computer/manipulator 
is,  the  human  operator  is  better  able  to  generate  commands  if  the  vocabulary 
is  suitably  constrained.  Probably  some  subset  of  natural  language  is  the 
appropriate  compromise. 
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For  example,  the  VERB-NOUN  syntax  is  a "natural"  sequence  that  has 
been  used  to  structure  operator  commands.  Most  computer  assembly  language 
is  written  this  way  {ADD  B;  STO  C;  CMP  D;  JMP  E).  The  Apollo  guidance 
computer  was  programmed  this  way  with  numbers  (15,  32;  03,  71;  ...).  Per- 
ceptronics' latest  keyboard  for  arm  control  (Shaket,  1977)  is  organized  around 
this  VERB-NOUN  syntax  (GOTO  5 DO;  INC  X (15)  DO),  adding  modifiers  (adverbs) 
and  terminators  (DO  represents  the  end  of  a command). 

Well -Structured  Programs.  Another  issue  related  to  the  natural  vs. 
constrained  trade-off  has  to  do  with  the  selection  of  control  structures 
(e.g.,  IF  [condition]  GOTO  [label]).  By  properly  constraining  the  choice 
it  may  be  possible  to  avoid  programs  which  are  impossibly  complex  and 
difficult  to  understand.  The  current  interest  in  "structured  programming" 
was  notivated  by  a concern  for  the  abilities  and  limitations  of  human 
programmers. 

Analogic  vs.  Symbolic.  This  issue  is  discussed  in  Section  5 in  regard 
to  the  hardware  for  command  inputs.  It  is  also  appropriate  here  in  the 
discussion  of  software,  because  any  language  for  manipulation  naturally 
has  analogic  descriptions  (of  actions,  forces,  directions,  orientations). 

Where  the  human  operator  is  required  to  make  such  specifications  the  appropriate 
communication  mode  ought  to  be  used.  For  example,  most  industrial  robots 
have  a training  mode  where  rather  than  specifying  positions  on  a keyboard 
as  numbers  (symbolic)  the  operator  drives  the  robot  manually  to  the  desired 
position  (analogic).  On  the  other  hand,  where  precision  is  required  the 
symbolic  mode  might  be  most  appropriate,  for  example,  in  programming  numeri- 
cally controlled  machine  tools.  Yet  even  for  numerically  controlled  tools, 
there  are  aspects  of  the  overall  planning  and  verification  of  tool  mot.jn 
where  the  analogic  and  pictorial  mode  is  more  appropriate  (Gossard,  1975). 

The  appropriate  form  of  command  hardware  may  depend  on  what  level  in  a 
hierarchy  of  task  abstraction  the  communication  takes  place.  Figure  6.1  illus- 
trates these  trade-offs.  Figure  6.2  elaborates. 
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FIGURE  6.1  COMMAND  HARDWARE  FOR  COMPUTER-AIDED  MANIPULATION  can  be  organized 
on  a task  hierarchy  with  different  communication  modes  appropriate 
at  different  levels.  At  the  highest  level  are  the  symbolic  commands 
(e.g.,  words,  programs,  labelled  positions  and  subroutines);  at 
the  lowest  level  are  the  analogic  commands  (e.g.,  directions,  amounts, 
positions  and  forces). 

Allowing  for  a range  of  communication  modes  has  been  proposed  as  an 
evolutionary  strategy  for  design  of  supervisory  control  (Verplank, 
1967).  The  symbol ic/analogic  dichotomy  may  have  to  do  with  the 
alternative  modes  of  human  knowing  and  thinking  (Verplank,  1976). 


TASK  HIERARCHIES 


man  machine 


FIGURE  6.2  ALTERNATIVE  TASK  HIERARCHIES  show  the  need.for  alternative  communi- 
cation modes. 

With  increased  automation,  the  communication  becomes  increasingly 
symbolic  (higher  on  the  hierarchy).  Properly  designed  supervisory 
control  should  allow  a range  of  communication  modes  (analogic  to 
symbolic)  for  different  levels  of  automation. 

Note  also  that  the  commands  may  sometimes  go  from  computer  to 
human  as,  for  example,  with  the  computer  keeping  track  of  a search 
strategy  or  assembly  sequence. 


! 
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Other  human  factors  Issues  have  already  been  discussed.  For  example, 
one  idea  is  that  for  sharing  control  the  computer  should  be  transparent; 
for  trading  control  it  should  be  apparent.  The  coding  of  symbolic  commands 
is  another  issue.  For  example,  should  special  purpose  keyboards  or  general 
purpose  keyboards  be  used,  single  keystrokes  or  multiple  keystrokes?  Is  a 
SEND,  DO,  or  CARRIAGE  RETURN  necessary?  Is  a back-space  or  delete  possible? 

There  are  many  principles  which  can  be  transferred  from  the  general 
domain  of  computer  programming.  For  example,  methods  of  dealing  with  both 
novice  and  expert  programmers:  HELP  commands,  menus  and  error  messages 
which  can  be  either  succinct  or  elaborate  , optional  abbreviation  of  both 
feedback  and  commands. 

Possibly  the  most  important  general  principle  at  this  early  stage  in 
the  development  of  supervisory  control  for  teleoperators  is  flexibility. 

As  experience  is  gained,  as  new  sensors  and  actuators  are  developed,  the 
trade-off  between  human  and  computer  control  will  shift.  A properly 
designed  supervisory  control  language  which  allows  communication  in  a variety 
of  levels  and  modes  will  be  ready  for  adaptation  and  evolution. 
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A brief  description  of  a thesis  by  T.L.  Brooks  in  progress  at  the 
Man-Machine  Systems  Lab  at  MIT  is  giver,  on  the  following  pages  as  an 
example  of  a supervisory  manipulator  system.  This  system  is  called 
SUPERMAN.  Figure  6.3  shows  the  general  relationships  between  the 
multiple  inputs  (keyboard,  dedicated  symbolic  keys,  and  analogic  inputs), 
the  computer  states  (STANDBY,  DEFINE,  EDIT,  EXECUTE,  and  TAKEOVER)  and 
the  control  modes  (RATE,  MIXED  MASTER/SLAVE  AND  RATE,  MASTER/SLAVE,  and 
COMPUTER  control). 

STANDBY  State  - When  the  computer  is  in  this  state,  control  resides  with 
the  main  program  and  the  operator.  By  pressing  the  proper  button  on  the 
control  console  the  user  can  enter  a particular  manual  control  mode  or 
another  computer  state  (see  Figure  6.4). 

Manual  Control  Mode  - A manual  control  mode  is  the  method  through  which 
the  user  analogically  interacts  with  the  arms.  A control  mode  is 
independent  of  the  state,  for  example,  the  control  mode  might  be  MASTER/ 
SLAVE  while  the  state  is  EDIT.  There  are  three  kinds  of  modes: 

1)  RATE  - The  individual  degrees  of  freedom  are  controlled 
through  rate  commands  by  switches  on  the  control  console 
and  a potentiometer  for  rate  adjustment.  Both  rate  and 
resolved-motion  rate  are  available. 

2)  MIXED  MASTER/SLAVE  AND  RATE  - The  master  acts  as  aspring- 
loaded  joystick  in  the  X,  Y and  Z axes,  giving  rate  commands 
to  the  X,  Y and  Z axes  of  the  slave  proportional  to  dis- 
placement of  the  master.  (The  rate  of  the  slave  arm  is  then 
reflected  in  the  force-feedback  level  which  the  operator 
feels  in  the  master.)  Both  rate  and  resolved-motion 

rate  control  are  available.  The  remaining  degrees  of  free- 
dom, the  left  and  right  elevation,  the  azimuth  and  the 
end-effector  are  controlled  in  a master-slave  mode. 


FIGURE  6.3  Computer  states  and  control  modes  for  SUPERMAN 


RATE 


SUPERMAN  Button  Box  (Brooks,  1978) 


3)  MASTER/SLAVE  - The  slave  arm  is  driven  to  duplicate  in 
position  the  action  of  the  master.  Any  force  felt  by  the 
slave  is  reflected  to  the  master  giving  the  operator  force 
•feedback  (i.e.,  proportional  to  position  disparity  between 
master  and  slave). 

DEFINE  - DEFINE  is  the  primary  state  through  which  the  operator  enters  a 
string  of  commands  to  be  executed.  Commands  are  entered  by  pressing  spe- 
cially dedicated  buttons  for  each  function.  All  of  the  buttons  used  in 
the  DEFINE  state  have  dual  functions  (see  Figure  6.4,  dual  function  buttons 
are  0-1 5) . 

EXECUTE  State  - As  the  title  implies,  the  string  of  commands  is  executed 
through  this  state.  During  the  execution  of  the  command  register,  if  the 
operator  desires  to  take  control,  there  are  two  methods  available.  The 
operator  can  take  immec  -,te  control:  (1)  by  pulling  on  the  appropriate 
control  stick  (i.e.,  the  MASTER  in  the  case  of  master-slave  or  MIXEE  MASTER/ 
SLAVE  AND  RATE  modes  or  the  rate  switches  in  the  RATE  mode),  or  (2)  by 
pressing  the  STOP  button  (all  action  ceases  after  the  STOP  button  has  been 
pressed  until  the  operator  signals  for  continuation  or  return  to  STANDBY). 
The  operator  can  execute  a string  of  commands  which  have  been  saved  as  a 
task  file  by  pressing  one  of  the  lighted  TASK  FILE  buttons.  The  operator 
also  has  the  option  of  executing  the  current  command  register  by  pressing 
the  EXECUTE  button.  This  allows  the  operator  to  define  a string  of  commands 
and  immediately  execute  them  to  determine  if  any  modifications  are  necessary 
After  the  operator  is  sure  the  command  string  performs  the  desired  function 
correctly  that  function  can  then  be  saved  as  a task  file  or  a named  file. 

TAKEOVER  State  - TAKEOVER  is  a transition  state  between  control  modes,  i.e. 
from  computer  control  to  the  control  mode  in  effect  before  the  EXECUTE  com- 
mand. Special  problems  result  during  this  state  due  to  the  mismatch 
between  the  master  and  the  slave  at  the  time  of  the  takeover.  The  diamond 
in  Figure  6.3  signifies  that  after  the  mismatch  has  been  dissolved,  the 
operator  has  the  option  of  moving  into  the  STANDBY  state  or  continuing  the 
EXECUTION  state. 
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The  dedicated-button  commands  associated  with  DEFINE  are: 


BUTTON 

Number 

0 


1 


2 


3 


4 


5 


6 


Command 

END 

Final  command  used  to  signal  completion  of  DEFINE  state. 
SAVE 

Used  to  save  the  command  register  on  the  disk  as  either 
a task  file  or  a named  file.  A task  file  can  be  recalled 
o.nly  by  one  of  eight  buttons  in  the  STANDBY  state,  whereas 
a named  file  is  saved  under  a user-designated  title  and 
can  only  be  recalled  by  the  same  name  through  the  GET  but- 
ton (5)  in  the  DEFINE  state. 

EDIT 

The  EDIT  command  allows  the  user  to  modify  the  command 
register.  The  following  options  are  available  through  the 
keyboard  after  entering  the  EDIT  state; 

a)  CHANGE  A LINE 

b)  INSERT  A LINE 

c)  DUPLICATE  A LINE 

d)  DELETE  A LINE 

e)  LIST  COMMAND  REGISTER 

f)  RETURN  TO  DEFINE 


2ND 

Used  to  enter  the  second  function  of  dual  command  keys. 

The  first  function  of  each  key  is  printed  in  black  letters 
above  the  button.  The  second  function  is  written  below 
the  button  in  gold  letters.  To  enter  a second  function 
command,  press  the  2ND  key  and  then  the  desired  second  com- 
mand. 

ERASE  LAST  LINE  [ERASE] 

Used  to  erase  the  last  entry  in  the  command  register, 

GET 

Used  to  retrieve  a named  command  file  from  the  disk. 

GET  asks  for  the  name  of  the  command  file  to  be  recalled 
and  then  locates  the  file,  reads  it  into  the  command  register 
(and  returns  to  DEFINE  state). 

RESET 

Used  to  initialize  the  necessary  internal  variables  and  the 
command  register  to  zero. 
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7 THROUGH  PATH  [TPATII] 

Records  the  present  position  of  the  arm  for  use  in  EXECUTE 
as  a through  point.  (A  through  point  is  a position  which 
the  operator  desires  the  arm  to  move  through  without  stop- 
ping, i.e.  non-zero  velocity  point.) 

8 INCREMENT  [DOF]  (XXXX) 

Makes  an  incremental  motion  in  the  desired  degree  of 
freedom  by  a selected  value.  The  user  enters  the  INCREMENT 
command,  then  the  degree  of  freedom  [DOF]] , adjusts  the 
desired  increment  (XXXX)  through  the  potentiometer  and 
presses  the  READ  POT  VALUE  button  directly  beneath  the 
potentiometer. 

9 IF  [DOF]  FORCE. GT. (XXXX) 

EXECUTE  NEXT  COMMAND 

If  the  force  level  in  the  desired  degree  of  freedom 
[DOF]  is  greater  than  the  level  set  by  the  operator  (XXXX) 
the  following  command  is  executed.  If  the  force  level  is 
less  than  the  level  set  by  the  operator,  the  command  immedi- 
ately follr  'ing  the  IF  FORCE. GT.  statement  is  skipped  during 
execution.  The  user  enters  the  IF  FORCE. Gl.  command,  then 
the  desired  degree  of  freedom,  adjusts  the  force  level  through 
the  potentiometer. 

10  GRASP  WITH  FORCE  (XXXX)  [GRASP] 

Tie  user  enters  the  GRASP  command  and  adjusts  the  force 
leel  through  the  potentiometer. 

11  DISCRETE  PATH  [DPATH] 

Records  the  present  position  of  the  arm  for  use  in 
EXECUTE  as  a stopping  point.  During  execution,  the 
slave  arm  is  moved  from  its  current  position  to 
the  recorded  position  with  zero  final  velocity. 

12  LABEL  [n] 

Labels  a position  in  the  command  register  which  can  be 
returned  to  through  a GOTO  command.  The  user  presses  the 
LABEL  button  and  then  the  number  [N]  of  the  desired 
label . 

13  GOTO  [N] 

GOTO  is  a conditional  command  which  moves  to  label  [N] 

unless  the  operator  signals  during  execution  to  change  the 
branch  to  [M]  by  pressing  a different  button.  To  enter 
the  command  the  operator  presses  the  GOTO  button  and  then 
the  number  [N]  of  the  label  to  which  GOTO  should  branch. 
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14 

15 


2ND  - 0 


2ND  - 1 


2ND  - 2 
through 
2ND  - 15 


OPEN 

Open  jaws. 

CONTINUOUS  PATH  [CPATHJ 

Records  the  position  of  the  master  manipulator  every  0.1 
second  for  use  in  EXECUTE.  A continuous  path  is  achieved 
by  interpolating  between  the  recorded  positions. 

ABSOLUTE 

Informs  the  execution  compiler  that  the  coimand  register 
is  to  be  executed  exactly  as  recorded  (see  RELATIVE).  The 
user  enters  the  absolute  command  by  pressing  the  2ND 
button  [#4]  and  then  the  ABSOLUTE  button  [#0]. 

RELATIVE 

Informs  the  execution  compiler  that  the  positions  in  the 
command  register  are  to  keep  the  same  relative  displacement 
with  respect  to  each  other,  but  are  to  be  transformed  so 
that  the  first  position  following  the  RELATIVE  conmand 
corresponds  to  the  position  of  the  slave  at  the  time  of 
execution.  A RELATIVE  command  can  be  cancelled  by  an 
ABSOLUTE  command,  with  the  result  that  only  the  positions 
between  the  RELATIVE  and  ABSOLUTE  coiranands  are  transformed. 
The  user  presses  the  2ND  button  [#4]  and  then  the  RELATIVE 
button  [#1J  to  enter  the  command  in  the  register. 


not  assigned. 


As  an  example  program  consider  a string  of  commands  to  take  a nut  off 
of  a bolt  and  put  it  in  a box.  This  program  can  be  broken  down  into  two 
major  sections;  one  removes  the  nut  and  the  other  places  it  in  the  box. 

Since  the  user  would  prefer  one  nut  removal  program  to  be  used  for  all  nuts 
regardless  of  the  orientation  of  the  nut,  a RELATIVE  command  should  obviously 
be  the  first  conmand  in  the  register  (the  RELATIVE  command  and  all  of  the 
following  commands  listed  above  are  used  under  DEFINE).  The  entire  command 
register  for  the  nut  removal  program  would  be  as  follows.  The  following 
general  format  will  be  followed  throughout  this  example: 

[BUTTON  PUSH] 

(POT  READINGS) 

"KEYBOARD  COMMANDS" 

COMPUTER  REPLIES. 
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1 [RELATIVE] 

2 [LABEL]  [1] 

3 [OPATH] 


4 [GRASP]  (200) 

5 [DPATH] 

6 [INCREMENT]  [Y]  (300) 


7 [IF  FORCE. GT.]  [Y]  (TOO) 


8 [GOTO]  [2] 

9 [GOTO]  [3] 


10  [LABEL]  [2] 

11  [INCREMENT]  [Y]  (-300) 

12  [OPEN] 

13  [GOTO]  [1] 

14  [LABEL]  [3] 

[SAVE]  ,:NUT-OFF" 


Place  the  slave  on  a nut  and  record 
that  position  by  pressing  the  DPATH 
button. 

Turn  the  end  effector  180°  and  record 
the  position. 

Increment  the  slave  by  300  counts  in 
the  direction  that  would  pull  the  nut 
off. 

If  the  force  is  greater  than  100  in 
the  Y direction,  the  nut  is  still  on 
the  bolt,  therefore  execute  the  next 
command . 

If  the  force  had  been  less  than 
100  in  the  Y direction,  the  nut  is 
free  and  this  command  would  be 
executed. 

Return  the  arm  to  position  before 
incrementing  in  #6. 

Release  the  nut. 

Return  to  LABEL  1 and  continue  turn- 
ing the  nut. 

End  of  the  first  part  of  task  - nut 
is  off. 

Save  command  register  as  the  named 
file  "NUT -OFF"  (typed  in  at  the  key- 
board) . 


The  second  part  of  the  task  requires  the  manipulator  to  place  the  nut 
in  a box.  The  entire  command  register  for  the  program  to  put  the  nut  in  the 
box  would  be  as  follows: 
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1 [ABSOLUTE] 

2 [TPATH] 


3 [DPATH] 


4 [OPEN] 

5 [TPATH] 

[SAVE]  "NUT-IN-BOX" 


The  box  would  always  be  in  the 
sanje  place. 

Move  the  slave  to  a position  just 
over  and  above  the  outside  edge  of 
the  box  and  record  this  position 
by  pressing  the  TPATH  button. 

Move  the  slave  to  a position  over 
the  center  of  the  box  and  record  the 
position. 

Enter  same  position  as  in  #2  by 
duplicating  line  2. 


At  this  point  the  operator  could  call  either  program  and  execute  it. 

The  NUT-OFF  program  would  simply  take  the  nut  off  and  return  control  to 
the  operator  as  soon  as  the  nut  was  free.  But  the  present  status  of  each 
file  (i.e.  a named  file)  requires  that  the  operator  type  in  each  name 
to  obtain  the  file  to  execute  it.  If  the  operator  performs  the  following 
commands  the  file  will  be  saved  as  a task  file  which  is  immediately  executed 
at  the  touch  of  a button: 

[GET]  "NUT-OFF" 

[GET]  "NUT-IN-BOX" 

The  computer  will  reply  by  stringing  the  two  files  together  as  one  file. 

Then  enter: 

[SAVE]  "TASK-FILE" 

and  press  the  button  which  will  retrieve  the  file  (e.g.,  button  #1).  To 
remove  a nut  and  put  it  in  the  box  the  operator  simply  presses  the  same 
button,  the  execution  compiler  transforms  the  first  half  of  the  register 
relative  to  the  position  of  the  slave  at  the  instant  the  button  is  pressed 
and  then  executes  the  program.  After  the  nut  is  removed  and  placed  in  the 
box  the  slave  returns  to  the  operator's  position  and  the  computer  relinquishes 
control . 
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7.  HUMAN  OPERATOR  PERFORMANCE 

This  section  reviews  data  comparing  the  performance  of  various  control 
modes  of  teleoperation  and  presents  some  of  the  attempts  to  model  and  predict 
performance.  The  focus  is  on  manipulation  and  not  on  vehicle  control  and 
navigation. 

Proper  evaluation  of  a manipulator  system  must  include;  1)  the  tasks 
which  can  be  accomplished;  2)  the  quality  (e.g.»  speed  and  accuracy)  with 
which  they  are  performed;  and  3)  the  "costs"  of  achieving  that  performance. 
Most  of  the  data  available  is  from  laboratory  experiments  on  a limited  set 
of  tasks  and  usually  completion  time  is  the  only  measure  of  performance 
reported. 

7 . 1 Performance  Measurement 

A recent  collection  of  approaches  to  the  performance  evaluation  of 
robots  and  manipulators  was  made  by  the  National  Bureau  of  Standards 
(Sheridan,  1976),  Much  of  what  is  discussed  there  is  appropriate  to  our 
concern  here  with  teleoperator  system  performance. 

The  manipulator  itself  will  be  characterized  by  a variety  of  object'''e 
mechanical  measures  such  as  reach,  work-volume,  strength,  slew  rate, 
number  of  degrees-of-f reedom,  range  of  motion  of  each  joint,  etc.  It  is 
much  more  difficult  to  evaluate  system  performance  which  includes  the  human 
operator  and  his  various  displays  and  controls.  The  most  direct  method 
is  to  measure  performance  on  a series  of  tasks.  The  tasks,  as  discussed 
in  Section  3 of  this  report,  may  span  a range  from  complete,  "real-world" 
tasks  representative  of  what  the  teleoperator  may  eventually  face  to  simple 
abstract  tasks  which  measure  some  detail  of  performance  capability. 

Physical  Measures  of  Task  Performance 

Popular  measures  are: 


7-1 


^ T*‘*ryr* '.'W-7 


T ine:  This  is  by  -far  the  most  commonly  used  measure.  It  is  easy  to  obtain 
and  directly  meaningful  (i.e.,  correlates  with  operating  costs). 
Accuracy : Some  tasks  require  a certain  threshold  accuracy  ("G0/N0G0"). 

Others  allow  variable  accuracy;  if  this  is  the  case  the  accuracy 
achieved  should  also  be  measured  and  reported. 

Errors : Error  rate  might  be  the  most  significant  difference  between  two 
systems.  Here,  error  is  taken  as  inadvertent  contact  or  direction 
reversals,  and  is  distinguished  from  accuracy. 

Force/Power:  Use  of  these  variables  as  performance  measures  may  make  some 
subtle  comparisons  possible.  Peak  force  levels  may  determine  mainte- 
nance needs.  Energy  usage  may  be  more  important  for  some  systems 
than  for  others  (e.g.,  untethered  vs.  tethered).  Force-feedback 
can  help  the  operator  minimize  manipulation  forces;  there  should  then 
be  less  "self-damage"  in  addition  to  allowing  better  cooperation 
between  the  vehicle  and  the  arm,  or  between  two  arms. 

Subjective  Measures  of  Task  Performance 

Much  of  the  actual  teleoperator  design  will  be  done  with  subjective 
evaluation  of  alternative  control  schemes.  Little  attempt  has  been  made 
to  collect  or  organize  these  opinions.  Some  attempt  to  devise  verbal 
scales  capturing  harc-to-quantify  aspects  of  performance  may  be  fruitful 
in  the  design  of  better  teleoperators.  Some  of  these  attributes  are: 
naturalness,  feel,  unobtrusiveness,  compliance,  dexterity,  programming 
ease,  flexibility,  stability.  Multi-dimensional  scaling  of  observer 
judgements  might  reveal  which  of  these  attributes  belong  on  the  same  or 
different  perceptual  dimensions.  Such  analysis  might  reveal  a simple  set 
of  standard  verbal  rating  scales,  or  at  least  the  dominant  attributes  in 
subjective  assessment  or  perhaps  the  need  to  re-educate  the  observers  making 
the  subjective  judgements. 

For  example,  we  might  develop  e 'really  there"  index  which  evaluates 
the  operator's  direct  sensing  of  the  remote  task  and  his  identification 
with  the  remote  hands  as  his  own.  Other  indices  might  include:  "inter- 
face transparency",  "responsiveness  and  controllability",  "sustained  work", 
"graceful  fail",  etc. 
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7 . 2 Human  Performance  in  Laboratory  vs.  At-sea  Situations 


Most  of  the  work  discussed  below  was  performed  using  laboratory  mani- 
pulators, for  reasons  of  economics  and  ability  to  control  key  variables. 

Hence  the  remainder  of  section  7 is  organized  on  the  basis  of  these  key 
variables  and  their  effects. 

When  real  mission  hardware  is  developed  it  is  certainly  desirable  to 
evaluate  it  under  semi -laboratory  conditions,  both  to  test  the  predictions 
of  previous  controlled  experimental  research  and  to  predict  at-sea  perfor- 
mance. 

An  example  is  provided  by  a recent  evaluation  of  the  Navy's  Work  Systems 
Package  (Bertsche  et  al , 1978),  described  in  section  5,  These  tests  utilized 
a variety  of  task  scenarios  done  both  in  the  laboratory  and  at  sea:  cutting 
openings  in  sheet  metal  structures,  removing  lightweight  objects  through 
the  cut  hole,  attaching  salvage  padeyes  with  drill -tap-bolt,  fasteners, 
drilling  bolt  holes  and  attaching  salvage  padeye  plates  with  multiple 
bolts,  rigging  a recovery  cable,  operating  valves,  identifying  and  recover- 
ing various  objects. 

Associated  performance  measurements  were  also  done  on  fourteen  component 
laboratory  tasks  which  "collectively  utilize  a representative  sample  of  all 
the  tool  suites  replicated".  These  were;  sample  retrieval,  acquire  tool, 
replace  tool,  acquire  bit,  replace  bit,  cut  rope  sample,  cut  cable  sample, 
brushing,  hooking,  valve  turning,  unbolting,  sawing,  drilling,  tapping. 

Each  of  these  in  turn  was  subdivided  into  "behavioral  actions"  (travel, 
alignment,  and  tool  use),  "therblig"  elements  such  as  those  discussed  in 
section  3,  the  smallest  elements  for  which  time  data  were  taken.  Four 
operators  performed  various  of  the  tasks  and  subtasks  under  both  direct 
and  video  viewing.  A major  finding  from  these  tests  was  that  operator 
experience  was  the  key  factor  in  coping  with  degraded  viewing  conditions. 

The  experimenters  claim  a valid  prediction  of  at-sea  performance  times 
from  laboratory  performance  on  comparable  scenarios.  Their  results  also 
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suggest  the  practical  advantages  of  having  a computer  perform  routine  tool 
changing  tasks,  the  motions  for  which  are  predictable,  as  well  as  controlling 
the  video  camera  to  follow  the  manipulator  end-point. 

7 . 3 Comparison  of  Manual  Control  Modes;  Effects  of  Rate  vs.  Position  Control 
and  Large  vs.  Small  Size 

In  general,  rate  control  is  slower  than  position  control  and  separate 
control  over  each  degree-of-freedom  is  slower  than  combined  control  (with 
a joystick).  Position  control  with  force-feedback  (master-slave  control)  is 
faster  than  without  force-feedback,  and  resolved-rate-control  (where  stick 
motion  corresponds  to  cartesian- coordinates  rather  than  joint-coordinates 
(achieved  through  computer  coordinate-transformations)  is  faster  than  the 
less  compatible  arrangements. 

Vertut  (1973)  has  compiled  a comparison  of  control  modes  for  handling 
radio-active  materials  (Figure  7.1).  Pesch  (1976)  has  done  the  same  for 
underwater  manipulators  (Figure  7.2).  Both  use  the  ratio  of  completion 
times  for  remote  vs.  direct  human  control  as  a summary  measure.  Mullen 
(1973),  in  a comparison  highlighting  resolved-rate-control,  uses  the  ratio 
of  completion  times  of  the  alternatives  to  master-slave  control  without 
force  feedback,  (Figure  7.3). 

There  are  circumstances  that  may  modify  these  conslusions.  With  a 
manipulator  which  cannot  match  human  arm  velocities,  the  advantages  of 
position-  over  rate-control  are  reduced.  Figure  7.4,  which  integrates 
experimental  results  of  four  different  investigators  who  compared  teleoperator 
to  human  operator  task  completion  times,  bears  this  out.  Conditions 
variously  used  were  resolved-motion  rate  control  and  master-slave  position 
control,  both  with  and  without  force  feedback.  Also,  if  extreme  precision 
is  required,  there  may  be  an  advantage  in  being  able  to  move  only  one 
joint  at  a time  to  avoid  the  inherent  cross-coupling  of  different  degrees- 
of-freedom  which  might  occur  with  a joystick  or  replica  (master)  controller. 
For  example,  Black  (1970)  found  with  time-delayed  manipulation  (using 
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FIGURE  7.1  EFFECTS  OF  CONTROL  MODE  ON  COMPLETION  TIME  can  be  compared  on 
the  basis  of  the  ratio  of  time  taken  to  do  the  task  with  mani- 
pulator divided  by  the  time  taken  by  a human  (Vertut,  1976). 

The  best  are  master-slave  manipulators  with  force-feedback  which 
which  are  2 to  10  times  slower  than  the  human  hand  depending  on 
the  complexity  of  the  task. 

Without  force-feedback  they  are  from  10  to  50  times  slower  than  the 
human  hand. 

Single-stick  rate-control  (RMRC)  is  faster  than  multiple  levers,  and 
proportional  rate  control  better  than  on-off-rate  control.  Some 
tasks  are  simply  impossible  without  the  compliance  that  force-feed- 
back provides. 

Currently,  only  one  undersea  manipulator  (Oceaneering  - G.E.,  Diver- 
equivalent-manipulator,  DEM)  is  master-slave  with  force-feedback. 
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FIGURE  7.2  TIME  RATIOS  FOR  UNDERSEA  MANIPULATORS  COMPARED  TO  DIVERS  have  been 
compiled  by  Pesch  (1976). 

This  illustration  summarizes  some  of  these  results,  indicating: 

1)  widely  differing  effects  of  control  mode;  2)  widely  differing 
effects  of  task  performed.  The  diver  in  this  case  was  operating 
under  relatively  ideal  conditions  and  the  teleoperator  was  relatively 
crude.  Thus  these  may  all  be  taken  as  "worse  case"  remote/diver 
ratios,  where  eventually  many  remote/diver  ratios  will  be  less  than 
unity.  Note  that  ratios  for  "elements"  are  larger  than  for  "tasks" 
Presumably  the  very  long  "elements"  in  Pesch's  study  constitute  small 
proportions  of  the  total  task  time. 


7-6 


RAT TO  OF  TASK  COMPLETION  TIME 


M.S.  « 1.00 


RMRC. /6 DOF 


R.C./6DOF 

RMRC/B.BOX 


R.C./B.BOX 


TASK  NUMBER 


FIGURE.  7.3  EVALUATION  OF  RESOLVED  MOTION  RATE  CONTROL  was  carried  out  by 
Mullen  (1973).  Task  completion  time  ratios  are  relative  to  the 
best  manipulator  control  mode  that  he  used  (master-slave  without 
force  feedback).  Depending  on  the  task,  he  found  RMRC  to  be  from 
2 to  8 times  slower  than  master-slave.  On  the  other  hand  RMRC  was 
generally  better  than  the  other  control  modes  he  tried,  the  worst 
of  which  was  rate-control  with  a button  box  which  is  the  mode  for 
most  underwater  manipulators. 


i 


; FIGURE  7.4  CONTROL  OF  LARGE  VS.  SMALL  MANIPULATORS.  By  combining  results  from 

several  investigators,  it  is  possible  to  show  that  the  time  advantage 
- of  master-slave  over  rate-control  is  reduced  for  larger,  slower 

| manipulators. 

i Wilt  (1977)  compared  "replica"  master-slave  control  with  force-feedback 

| (MSFF)  to  resolved  motion  rate  control  (RMPr)  for  a large  industrial 

^ manipulator  (14  ft.  reach,  24:1  force  ratio,  6.2:1  size  ratio). 

Mullen  (1972)  compared  RMRC  with  master-slave  control  for  a small 
manipulator  but  without  force-feedback  (MS).  Hill  (1977)  compared 
MS  to  MSFF  for  small  manipulators.  Vertut  (1975)  compared  large  (3:1) 
to  small  (1:1)  manipulators  using  MSFF. 
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master-slave  with  no  force-feedback)  that, while  waiting  for  visual  feedback, 
there  was  a drift  or  unintentional  input  by  the  operator  who  was  trying  to 
hold  the  master  still.  (The  effects  of  time  delay  are  discussed  in  more 
detail  in  section  7.5). 

Finally,  rate  control  might  be  of  advantage  over  position  control  in  a 
supervisory  scheme  if  control  is  being  traded  from  human  operator  to 
computer  and  back,  thus  minimizing  the  mismatch  of  operator  command  and 
computer  command  at  the  transition  (i.e.,  the  discontinuity  would  be  in  the 
rate,  not  in  the  position). 

There  are  other  factors  which  would  enter  into  the  choice  of  control 
mode  (e.g.,  rate  vs.  position  control)  such  as  the  space  and  weight 
required,  and  of  course  complexity  and  expense.  We  have  no  data  to 
present  here  other  than  the  task  performance  times. 

For  a task  of  transferring  lead  bricks  in  a nuclear  hot  cell,  Vertut 
(1976)  has  shown  that  the  time  to  move  one  block  (cycle  time)  depends  on 
the  type  of  hand  grip  used  and  the  ratio  of  force-feedback  used  (Figure  7.5). 

7.4  Positioning  Time  vs.  Accuracy:  Fitts'  Law. 

A convenient  summary  model  of  human  movement  speed  and  movement  accuracy 
is  given  by  what  has  come  to  be  known  as  Fitts'  law.  It  auantifies  two 
common  experimental  results:  (1)  movements  of  the  same  relative  accuracy 
(distance  A divided  by  tolerance  W)  appear  to  take  the  same  time,  and 
(2)  there  is  a 1ogarithmic  relation  between  movement  time  and  relative 
accuracy.  (Fitts,  1954;  Peterson,  1964),  I'sing  notions  of  uncertainty 
reduction  from  information  theory,  Fitts  defined  an  index  of  difficulty 
(1^)  to  include  the  ratio  (A/W)  and  the  logarithm  so  that  completion  time 
becomes  a linear  function  of  index-of-difficulty.  The  units  of  "difficulty" 
(with  1 og2 ) become  bits. 
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FIGURE  7.5  HANDGRIPS  AND  FATIGUE.  There  is  not  much  data  on  long  term 

performance  and  fatigue  effects  in  manual  control  of  manipulators. 

With  master-slave  force-feedback  control,  the  operator  must  apply  the 
same  forces  exerted  at  the  remote  end. 

Vertut  (1975)  found  that,  for  a test  task  of  transfering  lead  bricks, 
proper  hand  grip  design  will  reduce  completion  time  for  one  transfer 
but  that  fatigue  occurs  almost  as  soon  (approx.  50  transfers). 

Vertut  found  that  by  reducing  the  proportion  of  force  fed  back  to  the 
operator  (3:V)  he  can  work  nearly  three  times  as  long  without  fatigue. 
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Several  investigators  have  applied  Fitts'  law  to  describe  performance 
of  manipulator  systems.  Using  two  different  manipulators,  McGovern  (1974) 
applied  Fitts'  law  to  two  tasks,  "pick-up^peg"  and  "put-peg-in-hole", 
varying  distance  (A)  and  tolerance  (B-C ) , (see  Figure  7.6).  Roughly, 
completion  times  are  equivalent  for  the  two  tasks;  tasks  of  the  same 
difficulty  (1^)  take  the  same  time,  and  average  completion  time  (Tc)  is 
proportional  to  difficulty.  The  same  relationships  were  shown  to  hold  for 
the  two  manipulators  (Ames  and  SRI -Rancho)  and  for  the  unencumbered  hand. 

The  proportionality  of  time  and  difficulty  (with  different  slopes  for 
different  manipulators)  supports  the  notion  of  using  the  ratio  of  completion 
times  (manipulator  vs.  hand)  as  a key  performance  measure,  At  least,  the 
ratio  seems  to  be  constant  over  a range  of  task-difficulties. 

McGovern  found  the  same  ratio  of  completion  times  (manipulator  vs. 
hand)  for  two  different  tasks  and  a different  ratio  for  each  manipulator 
(Figure  7.7).  Pesch,  Vertut  and  Mullen  use  similar  ratios  to  compare 
control  modes,  but  find  different  ratios  for  different  tasks,  requiring 
a variety  of  tasks  for  manipulator  comparisons.  This  seems  an  appropriate 
outcome.  The  interesting  thing  from  McGovern's  work  is  finding  that 
there  are  at  least  two  tasks  with  the  same  ratio  and  that  the  ratio  is 
constant  over  a range  of  task  difficulties. 


Open-loop  positioning  accuracy.  Keele  (1969)  offers  a derivation 
of  Fitts'  empirical  law  based  on  a simple  assumed  model  of  open-loop 
(eyes  closed)  positioning  accuracy  and  a constant  time  (t.)  for  each  (discrete) 
feedback  and  movement.  The  assumption  here  is  that,  even  when  moving 
continuously,  the  human  operator  is  making  successive,  discrete  measure- 
ments. Movement  time  (Tffl)  is  then  simply  proportional  to  the  number  (n)  of 
"open-loop"  moves  (T  = nt).  If  each  movement  has  the  same  relative 

accuracy,  1-1 L . = 1<C  where  is  the  mean  absolute  distance  from  the 

/ 1 

center  of  the  target  after  the  ith  corrective  movement,  and  if  A 
is  the  starting  distance,  and  ^/z  is  the  distance,  where 

yj  is  the  width  of  the  target,  then 
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FIGURE  7.6  FITTS'  .INDEX  OF  DIFFICULTY  is  a useful  measure  of  task  difficulty. 

McGovern  (1974)  used  it  to  compare  two  different  manipulators  (both 
master-slave  without  force-feedback). 

Roughly,  over  a range  of  distances  (A)  and  tolerances  (B-C),  tasks  of 
the’ same  difficulty  (A/B-C)  take  the  same  time,  and  time  is  proportional 
to  the  index  of  difficulty  ( I ^ ) . 

Proportionality  of  time  and  difficulty  supports  the  notion  of  using 
the  ratio  of  completion  times  (manipulator  vs.  hand)  as  a key  performance 
measure.  (AMES  3:1 , RANCHO  10:1 ) , Tasks  other  than  peg-in-hole  will 
produce  different  ratios.  (McGovern,  1974). 
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FIGURE  7.7  PREDETERMINED  TIMF.  SYSTEMS  are  used  in  industry  to  predict  total 

task  times  from  known  times  for  motion  elements.  They  require  extensive 
calibration.  No  such  data  base  exists  for  manipulators. 

McGovern|s  (1974)  detailed  analysis  of  recordings  of  position  versus 
time  indicated  that  the  peg-in-hole  task  can  be  divided  into  two  phases 
as  in  MTM:  "reach"  and  "position".  The  "reach"  phase  is  from  start 
to  within  1.5  cm  of  the  hole.  "Reach"  time  is  linearly  related  to 
distance  and  independent  of  final  tolerance.  "Position"  time  is 
independent  of  distance  and  best  modelled  as  a logarithmic  function 
of  tolerance. 

This  data  was  later  used  by  McGovern  to  predict  the  value  of  an 
automatic  subroutine,  GROPE,  which  replaces  the  "position"  phase. 


Therefore, 


which  has  the  same  form  as  Fitts'  law  except  for  Fitts1  additive  constant, 
a.  Such  a constant  would  be  needed  to  fit  the  data,  for  example,  if  the 
first  or  last  move  were  slower  or  faster  than  the  rest, (or  the  initial  time 
to  decide  about  target  distance  were  included  ),  Keele  uses  t s 260  msec, 
and  K = .07  from  independent  experiments  and  predicts  a value  of  b = 70 
msec,  which  is  very  close  to  the  value  found  by  Fitts. 

7 . 5 Remote  Manipulation  with  Transmission  Delay 

Studies  of  delayed  auditory  feedback  showed  that  speaking  under  such 
circumstances  is  practially  impossible.  There  were  fears  expressed  that 
delayed  visual  feedback  (as  in  remote  manipulation  at  lunar  distance) 
would  make  manipulation  impossible.  Ferrell  ( 1 965 ) showed  that  it  is 
possible,  just  time-consuming,  The  human  operator  adopts  a "move-and-wait" 
strategy,  making  a succession  of  openjoop  moves  and  waits  for  feedback. 
Because  of  this  move-and-wait  strategy,  Ferrell  showed  that  it  is  fairly 
straightforward  to  predict  exactly  what  the  effects  of  increased  time- 
delay  are  going  to  be.  The  extra  time  is  simply  proportional  to  the 
number  of  waits  (or  open- loop  moves)  necessary  to  accomplish  the  task. 
Ferrell's  results  are  shown  in  Figures  7,8  and  7,9. 

Ferrell  looked  in  detail  at  the  issue  of  open-loop  movement  accuracy, 
using  a simple  two-degree-of-freedom-plus-grasp  master-slave  manipulator. 

He  found  that  the  standard  deviation  of  movement  error  was  not  a linear 
function  of  distance  (as  Keele  assumed),  and  that  a better  fit  was  variance 
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FIGURE  7.9  FERRELL'S  RESULTS  in  comparing  predicted  to  actual  number  of 

moves  and  predicted  to  actual  completion  time.  (Ferrell,  1965) 
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proporational  to  distance  (<^  55  jCA  ) . His  experiments  used  different  ; 

target  distances  for  successive  moves',  repetitive  moves  to  the  same  target 

distance  would  have  been  more  accurate.  With  this  assumption  (6^*  K.A  ) 

he  calculated  (using  Monte  Carlo  simulation)  the  number  of  open-loop 

moves  necessary  to  reach  a given  tolerance.  In  contrast,  Keele's  derivation 

neglects  probabilities  and  assumes  that  each  open-loop  move  is  always  the 

same  proportion  of  the  distance  to  the  target.  Ferrell's  theory  is  based 

on  a Gaussian  distribution  of  end-points  of  open-loop  moves  with  variance 

(not  standard  deviation)  proportional  to  distance.  The  number  of  moves 

predicted  is  roughly  proportional  to  Fitts'  index  of  difficulty  but  the 

curve  shifts  up  (more  moves)  for  greater  distance.  This  simple  model  of 

open-loop  uncertainty  accounted  quite  nicely  for  most  of  Ferrell's  results. 

Black  (1970)  studied  time-delayed  manipulation  (3  sec.)  with  a 6 degree-of- 
freedom-plus-grasp  manipulator  (Argonne  F.-2  master-slave  with  force-feed- 
back remoyed)  and  confirmed  Ferrell's  findings  of  the  move-and-wai t strategy. 

Black's  analysis  of  video  tapes  revealed  that  different  elements  of  the 
task  required  different  numbers  of  moves  (Figure  7.10).  Time  per  move 
averaged  five  seconds,  no  matter  what  portion  of  the  task  was  considered. 

This  is  explained  by  the  majority  of  that  time  being  spent  waiting  (3 
seconds  per  move) . 

Those  portions  or  elements  of  the  task  which  required  the  greater  number 
of  moves  required  a larger  percentage  of  total  task  completi on-time  with 
the  delay  as  compared  to  without  the  delay.  This  result  is  illustrated 
in  Figure  7.11. 

More  recently  Thompson  (1977)  measured  task  completion  time  with  varying 
degrees  of  task  constraint  (using  Hill's  tasks.  Figure  3.12)  and  with  two 
different  manipulators  plus  direct  human  manipulation.  The  results  are 
given  in  Figure  7.12.  Thompson  also  studied  the  effects  of  loop  time  delay 
on  performance  on  the  same  set  of  tasks,  using  the  NASA  Ames  master-slave 
manipulator  (see  results  in  Figure  7.13). 

Starr  (1976,  1978)  compared  position  vs.  rate  control  with  time  delay. 
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FIGURE  7.10  TIME-DELAY.  Master-slave  manipulation  with  a 3-second  time-delay 

is  characterized  best  by  the  move -and- wait  strategy  used.  Completion 
time  is  dominated  by  the  amount  of  time  spent  waiting. 

Black  (1970)  counted  the  number  of  waits  for  feedback  from  video 
tapes.  No  matter  which  task  element  was  being  performed  (get, 
transport,  position)  completion  time  is  proportional  to  number  of  waits. 
This  emphasizes  the  importance  of  open-loop  movement  accuracy  from 
which  the  number  of  waits  can  be  predicted  (see  Figure  7,8). 
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FIGURE  7.11  ELEMENT  TIME  PROPORTIONS  change  with  time-delay  depending  on  the 
number  of  movements  require^.  "Transport"  (1-2  moves)  becomes  a 
smaller  fraction,  "Get"  (4-6  moves)  stays  about  the  same,  and 
"Position"  (4  - 10  moves)  becomes  a larger  proportion  of  the  total 
task  time.  (Black,  1970). 
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FIGURE  7.12  VARIABLE  DEGREE -OF-CONSTRAINT  TASKS  were  used  by  Hill  (1976)  to 
compare  two  manipulator  systems  with  the  human  hand.  The  tasks 
are  shown  in  Figure  3.12.  Thompson  (1977)  used  the  same  tasks 
for  a study  of  time-delay  (Figure  7.11). 
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FIGURE  7.13  -EFFECTS  OF  TIME-DELAY  ON  COMPLETION  TIME  are  compared  for  six 
different  tasks.  This  is  a replot  of  data  from  Thompson  (1977) 
to  show  that  there  is  a linear  increase  in  completion  time  as  a 
function  of  time-delay.  The  slope  of  the  line,  as  shown  by 
Ferrell,  is  simply  the  number  of  open-loop  moves  necessary  to 
complete  the  task  (ranging  from  6 for  the  0 d.o.c.  task  to  27  for 
the  5 d.o.c.  task) . 
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He  found  that  over  a range  of  time  delays  up  to  3 seconds  the  advantage  of 
position  control  over  rate  control  is  reduced  as  time  delays  get  larger. 

Delayed  force- feedback.  The  advantage  of  force-feedback  in  master-slave 
manipulation  may  turn  to  difficulty  with  a time-delay  in  the  control  loop. 
Ferrell  (1966)  has  shown  that  if  forces  are  fed  back  to  the  hand  which  also 
provides  the  positioning  command  to  the  manipulator,  they  will  tend  to  move 
the  operator's  hand.  If  the  delay  and  the  rate  at  which  feedback  force 
changes  with  the  position  of  the  remote  harid  are  great  enough,  a manipula- 
tor can  become  unstable.  Unstable  movements  can  easily  be  avoided  with' 
purely  visual  feedback  since  the  operator  can  attend  to  the  information 
selectively;  and  error  indication  need  not  result  immediately  in  a response 
by  the  operator.  Ferrell  suggests  displaying  the  delayed  force  information 
to  the  other  hand  or  having  some  mechanism  at  the  remote  end  for  limiting 
applied  forces. 

A careful  distinction  should  be  made  between  the  transportation  time 
for  a signal  and  the  transmission  time  for  a complete  message.  The  signal 
transportation  time  via  electro-magnetic  radiation  is  the  distance  divided 
by  the  speed  of  light.  Round  trip  to  the  moon  for  transport  time-delay 
is  about  three  seconds.  This  was  the  basis  for  the  choice  of  time-delay  in 
many  of  the  experiments  on  time-delayed  manipulation  which  NASA  has  spon- 
sored. 

The  transmission  time  for  a message  depends  on  trie  capacity  of  the 
channel  (bits  per  second)  and  the  information  in  the  message  (bits).  En- 
coding (for  error  correction  redundancy) and  validation  may  further  increase 
the  time  for  complete  transmission  of  a message.  For  example,  a high 
resolution  T.V.  image  requires  3 million  bits;  at  10,000  bits/second  (a 
good  underwater  voice  channel)  this  would  require  300  seconds  (i.e.,  5 
minutes)  per  frame.  Remote  manipulation  undersea  through  such  a data  link 
will  be  limited  by  the  slow  frame  rate,  net  by  the  transport  time-delay. 

The  laboratory  results  for  time-delayed  manipulation  may  or  may  not  be 
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relevant  to  the  situation  of  slow  frame  rate,  which  may  characterize  future 
undersea  teleoperator  systems.  The  move-arid-wait  features  cf  self-paced 
perform;. nee  will  probably  be  the  same  but  the  difference  between  having  a 
moving  picture  (time-delayed)  and  a still  picture  (slow-frame-rate)  may 
produce  very  different  performance.  Ferrell's  use  of  an  underlying  relation- 
ship between  open-loop  movement  distance  and  accuracy  may  be  the  key  to 
modelling  and  predicting  performance.  For  example,  the  trade-off  between 
frame-rate  (frames  per  second)  and  resolution  (bits/frame)  might  be  pre- 
dicted through  such  a simple  model  which  includes  less  accuracy  at  lower 
resolution. 

7,6  Remote  Vehicle  Control  with  Transmission  Delay  arid  Slow-Frame-Rate 

Section  5 describes  a predictor  display  system  for  coping  not  only 
with  transmission  delay  but  also  with  slow  frame  rate.  Verplank  (1978) 
explored  the  effects  of  such  a predictor  display  for  vehicle  control.  An 
interactive  simulation  was  written  on  an  Interdata  70  computer  and  Imlac 
graphic  display.  A random  terrain  was  generated  and  displayed  in  perspective, 
updated  every  8 seconds,  to  simulate  the  pictorial  information.  A moving 
predictor  symbol  was  generated  representing  the  vehicle  as  a square  in 
perspective.  Two  straight  ridges  were  added  to  the  random  terrain  to  serve 
as  a test  course.  (Figure  7.14). 

The  simulated  vehicle  was  controlled  by  the  operator  with  a spring- 
centered  2-degrees-of-freedom  joystick.  The  dynamic  response  of  the  vehicle 
was  a simple  integration  with  forward  speed  proportional  to  fcrward-back 
position  of  the  stick  and  turn-rate  proportional  to  left-right  position  of 
the  stick.  The  vehicle  was  always  the  same  height  above  the  terrain 
(simulating  automatic  altitude  hold).  No  disturbances  such  as  currents 
were  simulated.  Also,  it  was  found  important  to  have  a good  detent  and 
dead-zone  on  the  stick  to  avoid  inadvertent  commands. 

A stationary  "table"  was  drawn  to  indicate  where  the  next  picture  was 
to  come  from  while  the  "real-time"  predictor  continued  to  move  in  response 
to  the  operator's  commands  (Figure  7.14),  Dotted  lines  were  added  to  this^ 
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FIGURE  7.14  SIMULATION  EXPERIMENTS  WITH  PREDICTOR  DISPLAYS  were  performed  by 

5/erplank  (1978).  Slow-frame-rate  pictures  (8  seconds  per  frame)  were 
simulated  by  computer-displayed  terrain.  The  path  to  be  followed 
was  a ridge  in  the  terrain. 

A moving  predictor  symbol  (perspective  square)  was  superposed  on  the 
static  picture  of  terrain.  The  point  from  which  the  next  picture 
was  taken  was  indicated  with  a "table11  (square  with  four  legs) 
and  the  field  of  view  was  shown  with  dotted  lines. 


table  to  indicate  the  field  of  view.  This  reduced  the  considerable  con^ 
fusion  about  how  the  picture  was  expected  to  change  and  served  as  a guide 
for  keeping  the  vehicle  within  its  own  field  of  view,  which  is  the  best 
strategy  for  using  this  kind  of  predictor  on  the  pictorial  display. 

A typical  trajectory,  without  the  predictor,  is  shown  in  Figure  7.15a. 
The  dotted  lines  represent  + one  terrain-unit  from  the  ridge.  The  circles 
represent  the  vehicle's  position  every  2 seconds,  V's  represent  the  field 
of  view  of  each  picture  sent.  Quite  often  there  is  no  movement  between 
successive  dots  (2  secs.)  or  successive  pictures  (8  seconds) 

Only  with  extremely  slow  speed  was  it  possible  to  keep  track  of  the 
ridge,  Approximately  five  minutes  and  40  pictures  were  required  to 
traverse  just  one  of  the  ridges  (half  the  course).  This  is  shown  in 
Figure  7.15b, 

With  the  predictor  symbol,  practially  continuous  motion  was  possible, 

A typical  trajectory  is  shown  in  Figure  7.16a,  The  course  was  completed 
in  3 minutes  and  23  pictures. 

A typical  trajectory  in  request  mode  (where  pictures  are  only  sent  by 
operator  request,  as  explained  in  Section  5,3)  is  shown  in  Figure  7.16b. 
Compared  to  periodic  mode,  the  time  is  about  the  same  but  the  number  of 
pictures  used  is  one-half  to  one-third;  velocities  are  higher  but  there 
is  a wait  for  10  seconds  as  each  picture  is  taken  and  sent, 

For  the  conditions  studied  (T  = 1 sec.,  S = 8 sec,)  manual  control  is 
not  feasible  without  display  aids  such  as  the  predictor  symbol.  The  request 
mode  is  preferred  as  it  seems  to  avoid  confusion  and  reduce  the  number  of 
pictures  necessary. 

7 . 7 Viewing  Conditions  for  Remote  Mail ipulation 

For  remote  manipulation  the  primary  mode  of  feedback  to  the  operator  is 
visual.  The  only  means  presently  used  is  television.  (For  a manned  sub- 
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FIGURE  7.16  SIMULATION  RESULTS  WITH  PREDICTOR  DISPLAY  show  good  remote  vehicle 
control  in  spite  of  slow  frame  rate  (8  seconds/frame)  and  time-delay 
(1  sec).  In  periodic  mode,  (a),  one  picture  is  sent  every  8 seconds 
in  request  mode,  (b)  pictures  are  sent  only  upon  the  operator's 
request.  Travel  times  are  equivalent  for  the  two  modes;  request 
mode  uses  fewer  pictures  but  requires  a 10  second  pause  in  vehicle 
motion  as  each  picture  is  sent.  (Verplarik,  1978). 


mersible,  direct  yiewing  through  a port  is  also  used.) 

"i 

One  of  the  natural  things  to  try  for  remote  viewing  is  stereo.  On  the 
whole,  results  have  been  disappointing.  There  are  many  practical  problems.  • 

Stereo  television  is  more  complex  and  expensive;  usually  it  is  less  reliable. 

The  resolution  can  be  poorer,  and  the  viewing  apparatus  an  encumberance. 

There  are  many  subtle  errors  possible  with  mismatched  lenses,  and  non-  1 

linearity  in  image  transfer,  (The  distortion  might  not  be  distinguishable 
from  depth  dues),  Control  over  focus  and  convergence  is  complex  and 
inconvenient. 

There  is  significant  learning  for  manipulation  through  either  stereo 
or  mono- television.  In  fact,  the  differences  between  mono  and  stereo 
(task  completion  time  and  errors)  appear  to  be  reduced  with  practice 
(Pepper,  1977).  See  Figure  7,17a. 

The  possible  advantage  of  stereo  over  mono  viewing  may  be  greater  for 
turbid  water.  The  particles  intervening  between  object  and  lens  will  be 
different  in  the  two  views.  The  human  observer's  natural  correlation  of 
the  two  views  serves  to  "filter”  the  "noise"  of  uncorrelated  particles. 

The  predicted  effects  of  turbidity  are  shown  in  Figure  7,17b.  •• 

There  are,  of  course,  other  methods  of  picking  up  depth  cues;  a second  . ' 

view  from  another  camera  (or  from  the  same  camera  after  moving  it),  the 
differing  amounts  of  intervening  turbidity,  varying  amounts  of  illumination,  \ 

shadows,  markings  on  objects . Color  may  or  may  not  be  an  advantage;  the 
added  cues  are  at  the  expense  of  the  sensitivity  and  resolution  possible  j 

with  high  quality  black  and  white  cameras  and  monitors. 

NASA  sponsored  a comprehensive,  multivariate  laboratory  study  of  ! 

viewing  conditions  for  remote  manipulation  in  space  (Freedman,  1977). 

The  results  are  summarized  in  Figure  7.18,  The  only  variables  which  affected  ! 

task  completion  time  significantly  are  the  task,  arm  speed,  and  the  TV 
system  (mono-  B+W  more  time  than  stereo).  On  the  other  hand,  when  position  j 

error  is  of  concern,  the  two-view  TV  system  is  better  than  all  the  others 
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FIGURE  7.17  STEREO-  VS.  MONO-VIEWING  was  studied  by  Pepper,  et  al . (1977)  for  a 
simple  manipulator  positioning  task  in  a fairly  complex  scene  which 
simulated  underwater  conditions.  Stereo  was  always  better  than  mono 
especially  in  terms  of  contact  errors,  but  the  differences  were 
reduced  with  practice. 

The  advantage  of  stereo  might  be  greater  in  turbid  water.  This  is 
illustrated  with  the  hypothetical  curves  on  the  right. 
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FIGURE  7.18  VIEWING  FOR  REMOTE  MANIPULATION  IN  SPACE  was  studied  in  a multi- 
variable  laboratory  simulation.  (Freedman,  et  al . 1977).  Time 
and  error  averages  are  shown  for  each  independent  variable  in 
comparison  to  the  grand  mean  (horizontal  line).  The  difference 
necessary  for  significance  is  shown  with  the  dotted  vertical  lines 
(where  there  is  none  there  were  no  significant  differences). 

TV  system  rankings  were  made  for  over-all  performance  and  burden 
(cost,  weight,  etc.).  B&W  mono  is  best  (lowest  combined  rank)  when 
burden  is  more  critical;  2-views  are  better  when  performance  is 
more  critical  than  burden. 
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Figure  7.18),  (Means  not  separated  by  more  than  the  length  of  the  dotted 
lines  are  not  significantly  different;  for  variables  with  no  dotted  line 
shown,  there  are  no  significant  differences.) 

Freedman's  summary  recommendations  considered  two  measures  of  each 
TV  system-:  an  oyer-all  performance  rank  (smaller  rank  is  better)  and  a 
burden  rank  (smaller  burden  is  also  better)  based  on  cost,  complexity, 
reliability,  weight,  etc.  For  example,  the  two-view  system  ranked  best 
for  performance  but  worst  for  burden,  The  black  and  white  mono  system  was 
ranked  worst  for  performance  but  best  in  terms  of  burden,  The  choice 
of  viewing  system  depends,  then,  on  the  relative  weighting  of  performance 
and  burden.  If  cost  is  no  object,  then  the  two-view  system  should  be 
chosen.  If  burden  must  be  minimized  then  the  mono  system  is  best.  There 
may  be  a small  range  in  between  where  either  color  or  stereo  is  the 
best  choice. 

7.8  Use  of  Proximity  Sensors 

Bejczy  (1976)  has  demonstrated  the  use  of  proximity  sensors  with 
display  to  the  operator  for  remote-manipulation  (See  Figure  7.ig).  One 
of  the  important  variables  that  determines  the  value  of  proximity  sensors 
is  the  viewing  conditions.  For  a simple  block  stacking  task  he  found 
that  either  "front"  or  "down"sensors  improved  performance  over  mono  or 
stereo  viewing  alone  but  that  a two-view  system  showed  the  same  improve- 
ment without  proximity  sensors.  That  is,  you  don't  need  the  proximity 
sensor?  if  you  can  see  well  enough.  It  is  interesting  that  when  both 
"front"  and  "down"  sensors  were  used  simultaneously  performance  was  worse 
than  when  either  was  used  alone.  This  is  probably  due  to  the  confusion 
of  the  auditory  display  used.  Bejczy  is  now  working  on  a visual  display 
of  proximity  sensor  information  (Bejczy  and  Paine,  1977). 

When  the  proximity  sensor  information  is  used  by  the  computer  rather 
than  by  the  operator  certain  tasks  can  be  accomplished  more  quickly.  For 
a blind  positioning  task,  Bejczy  (1976)  found  that  the  computer  could 
stop  the  arm  more  quickly  and  more  accurately  than  could  the  human  operator 
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FIGURE  7 19  PROXIMITY  SENSORS  have  been  developed  at  the  Jet  Propulsion  Laboratory 
(Bejczy  1976).  The  results  for  an  auditory  display  of  proximity 
information  are  shown  at  the  top.  Subsequently,  a visual  display  was 
developed  (Bejczy  and  Paine,  1977)  which  allowed  greater  accuracy. 

Computer-aided  control,  where  the  computer  stopped  the  arm  at  a 
fixed  proximity,  improved  both  time  and  accuracy. 
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using  the  auditory  display  of  proximity  information  (Figure  7.19),  It 
remains  to  be  seen  whether  this  laboratory  demonstration  will  lead  to 
similar  applications  undersea, 

7 . 9 Evaluation  of  Computer-Aids;  Supervisory  Control 

The  computer-aids  to  manipulation  discussed  in  chapter  6 were  divided 
into  sharing  and  trading  modes. 

The  only  extensive  evaluation  of  the  sharing  mode  has  been  for 
resolved-motion-rate-control  where  the  computer  does  coordinate  trans- 
formations between  the  operator's  commands  (in  room-  or  hand-coordinates) 
and  the  manipulators  individual  joint  velocities.  The  results  have  been 
summarized  in  section  7.3,  showing  RMRC  to  be  the  best  form  of  rate  control. 

There  have  been  several  demonstrations  of  the  trading  mode  (Barber, 

Hill,  Freedy,  Bejczy)  but  little  data  has  been  accumulated  to  show  under 
what  circumstances  the  trade  to  computer  control  is  of  advantage. 

McGovern  (1974)  made  a detailed  study  of  direct  human  control  in  a peg- 
pick-up  task  to  predict  under  what  circumstances  an  automatic  pick-up 
program  (GROPE)  would  be  taster.  Rather  than  describing  human  performance 
with  Fitts'  law  he  found  that  a more  convenient  and  just  as  accurate 
representation  is  to  separate  motion  into  a "reach"  phase  (to  within  1/2" 
of  the  block)  and  a "position"  phase  (the  rest).  Reach  time  depends  on 
distance  and  is  independent  of  tolerance.  "Position"  time  depends  on  toler- 
ance (B-C)  and  is  independent  of  distance  (See  Figure  7.7  ).  The  GROPE 
subroutine  was  invented  and  demonstrated  by  Hill  (1973).  The  manipulator 
jaw  is  equipped  with  touch  sensors.  The  GROPE  subroutine  takes  over  control 
when  one  of  fingers  touches.  It  then  increments  (a  fixed  amount,  ) until 
centered  on  the  peg.  Thus,  GROPE  replaces  the  "position"  time. 

McGovern  showed  that  GROPE  should  improve  performance  only  over  a 
small  range  of  tolerances  depending  on  what  the  movement  increment  is. 

(See  Figure  7.20).  IF  the  increment  is  larger  than  the  tolerance 
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FIGURE  7.20  PREDICTED  PERFORMANCE  OF  "GROPE"  SUBROUTINE.  McGovern  (1974)  compared 
the  estimated  time  for  computer  controlled  positioning  (GROPE)  to 
that  for  manual  control  (Figure  7.7)  and  showed  that  there  is  only  a 
small  range  of  tolerances  (B-C)  where  GROPE  will  improve  performance. 

The  human  vs.  computer  tradeoff  also  depends  on  the  movement  increment 
(AI ) that  GROPE  uses. 

If  A I is  larger  than  the  tolerance, GROPE  overshoots;  if  A I is  too  small, 
GROPE  is  slower  than  the  human  operator. 
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the  jaw  will  step  too  far,  missing  the  peg,  If  the  increment  is  too  small, 
the  computer  control  will  take  too  many  steps  and  be  slower  than  the 
human,  This  theoretically  leaves  a small  region  where  GROPE  may  improve 
performance.  Ot'  r circumstances  (e.g,,  time-delay,  poor  picture)  may 
modify  these  conclusions;  also, other  sensors  or  more  clever  program,,  may 
do  better  than  GROPE, 

The  importance  of  McGovern’s  work  was  demonstration  that  a detailed 
look  at  human  performance,  with  the  appropriate  summary  measures, wil 1 
be  necessary  in  making  decisions  about  when  and  how  much  computer-aiding 
to  use  The  work  of  Wernli,  et.  al . (1978)  on  the  Work  Systems  Package 
is  similarly  appropriate  (see  section  7,2), 

For  several  years,  Freedy,  Weltman  and  others  at  Perceptronics  have 
experimented  with  computer-aids  to  manipulation.  One  method  (Freedy  et  al , 
1971)  used  was  a learning. system  (ACS)  which  observed  the  human  operator's 
motion  of  the  manipulator,  If  the  motion  was  repetitive  enough,  the 
computer  could  make  (with  varying  degrees  of  confidence)  a prediction 
of  what  the  next  motion  was  going  to  be,  take  over  control  from  the 
operator,  and  execute  the  most  likely  trajectory.  This  was  an  interesting 
demonstration  of  computer  power  at  simple  pattern  recognition  but  not 
much  use  for  accomplishing  practical  manipulation  tasks.  The  operator  is 
seldom  interested  in  repeating  the  "average"  of  a series  of  motions; 
where  repetition  is  necessary,  it  is  easier  for  the  operator  to  explicitly 
show  the  computer,  with  one  demonstration,  what  is  to  be  repeated  rather 
than  having  the  computer  try  to  figure  it  out  from  repeated  demonstrations. 

Recent  work  by  Perceptronics  is  on  explicit  programming  where  control 
can  be  traded  between  human  and  computer.  Arm  positions  can  be  recorded  and 
returned  to  with  the  push  of  a button  (actually  a series  of  key  strokes). 
Laboratory  results  to  date  have  not  shown  unequivocally  the  advantage  of 
trading  control  with  the  computer. 

The  usefulness  of  such  a "go-to-point"  automatic  subroutine  depends 
on  the  number  of  times  the  point  is  used  and  the  comparable  time  for  doing 
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< it  manually,  the  time  needed  to  define  the  point  and  record  the  subroutine, 

and  the  time  it  takes  to  invoke  the  subroutine  (number  of  key  strokes). 

! Perceptronics  found  one  task  (valve  turning)  where  "go-to-point"  was  an 

advantage,  but  only  if  the  necessary  points  were  pre-defined.  Their 
results  are  shown  in  Figure  7.21 . The  amount  of  movement  time  is  consid- 
erably reduced  by  computer  control  as  compared  to  direct  rate  control 
(joystick  and  toggle  switch),  But  the  added  time  for  definition  and 
invocation  make  computer-aided  control  no  faster,  If  the  "go-to"  points 
are  pre-assigned,  then  there  is  an  advantage  to  computer-aiding. 

For  a more  elaborate  "integrated  maintenance  task"  they  found  that 
there  was  no  significant  advantage  to  the  go-to-point  subroutine.  What 
they  concluded  was  that  the  task  was  net  repetitive  enough  (did  not  use 
the  pre-recorded  points  enough  times),  and  that  the  schemes  used  for 
definition  and  invocation  were  not  as  convenient  as  they  could  be,  A 
revised  design  for  keyboard  and  syntax  has  resulted  (Shaket,1977). 

To  properly  predict  and  plan  for  the  use  of  supervisory  control  for 
teleoperation,  a full  understanding  and  good  data  base  for  direct  manual 
control  will  be  of  value.  Automatic  control  should  be  compared  with 
not  just  one  manual  control  mode  such  as  fixed-rate  switch-control  but 
with  the  many  alternative  manual  control  modes  such  as  resolved-motion 
rate  or  position  control.  One  of  the  difficult  choices  may  be  between 
force-feedback  and  computer  control  (between  "augmentation"  and  "automation"). 
If  control  is  to  be  traded  from  manual- to  computer-control  and  back,  then 
the  particular  form  of  manual  control  may  be  crucial. 
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FIGURE  7.21  TIME  TO  TEACH  AND  CALL  AUTOMATIC  SUB-ROUTINES,  when  added  to 

movement  time,  may  make  computer-aided  manipulator  control  no  faster 
than  direct  manual  control. 

For  a repetitive,  valve  turning  task  the  "go-to-point"  routines 
only  reduced  task  time  when  the  points  were  pre-defined.  (Berson, 
et  al.,  1977). 


7-37 


8.  MODELS  OK  SUPERVISORY  CONTROL 


8.1  Models  and  them  Uses 


A model  is  a representation  of  some  part  of  the  real  world  rendered  as  a 
scaled  (down  or  up)  replica  in  three  dimensions,  or  a drawing  or  diagram  in 
two  dimensions,  or  a set  of  equations  or  symbolic  statements,  or  a computer 
program.  The  model's  purpose  is  to  characterize  or  portray  certain  salient 
variables  in  the  real-world  situation  and  the  relationships  between  them,  with 
respect  to  magnitude  and  time. 

Models  are  used,  then,  to  describe  observed  events  and  to  predict  future 
events.  Descriptive  models  are  those  whose  function  is  to  describe,  as  concise- 
ly as  possible,  relationships  between  experimentally  observed  events,  with  an 
aim  to  predicting  future  events  in  similar  situations,  and  without  a priori 
regard  for  a mechanistic  or  teleological  (goal  seeking)  b as is  for  the  relationships. 
The  simplest  "black-box"  or  input-output  description  is  the  best.  A normative 
model  also  seeks  to  describe  and  predict  experimental  events,  but  it  starts  from 
the  premise  that  a certain  mechanism  is  at  work,  or  in  the  case  of  a teleological 
system,  that  certain  goals  are  sought,  or  that  a certain  "objective  function"  is 
the  basis  of  optimization  or  compromise  among  performance  variables.  Thus,  unlike 
the  descriptive  model,  the  normative  model  tries  to  show  the  degree  to  which  ex- 
perimentally observed  behavior  resembles  that  produced  by  a given  theoretical  norm. 

Least  squares  curve  fitting,  factorial  analysis  of  variance,  state  deter- 
mined Markov  (transition  probability)  models,  information  transmission  models, 
describing  functions  and  other  identification  models  of  control  systems  all  tend  to 
be  of  the  descriptive  model  sort.  The  theoretical  structures  used  ar ; adapted 
for  describing  what  happened  and  from  this  predicting  what  will  be.  They  make 
no  presupposition  about  underlying  mechanism  or  purposiveness  of  behavior.  In 
contrast,  Bayesian  models,  signal  detection  models  and  optimal  control  models  arc 
normative.  All  constitute  ideal  norms  or  perfect  mechanisms  of  behaviot  with 
respect  to  which  human  ot  physical  system  behavior  may  be  compared. 

It  is  commonly  appreciated  that  the  choice  of  model  type  is  made  mostly  by 
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art  and  not  science,  depending  upon  the  modeler  and  the  tastes  of  the  community  of 
persons  to  whom  he  wishes  to  communicate  his  research.  It  is  not  so  well  under- 
stood that  th*.  criterion  variables  with  respect  to  which  models  are  "fit"  to  ex- 
perimental events  are  similarly  chosen.  Below  are  listed  different  kinds  of 
criterion  variables  which  are  commonly  used  to  fit  models  to  experimentally  de- 
rived data  from  man-machine  systems: 

1.  degree  to  which  model's  behavior  produces  same  overall  or 
final  "success"  as  observed  system 

2.  degree  to  which  model's  behavior  produces  same  success 
in  component  tasks 

3.  degree  to  which  model  makes  same  set  of  responses,  in- 
dependent of  time  or  order 

4.  degree  to  which  model  produces  same  sequence  or  trajectory  of 
response  states,  independent  of  time 

5.  degree  to  which  there  is  a response  correspondence  at  each 
point  in  time 

6.  degree  to  which  subjective  ratings  by  human  observer  are  same 
for  model  and  observed  system 

The  dilerrana  is  that,  given  the  same  basic  model  and  same  empirical  data,  dif- 
ferent parameter  coefficients  will  be  best  fits  depending  on  which  fitting  cri- 
terion is  chosen.  Ideally  a model  should  provide  a perfect  fit  to  the  correspon- 
ding experimental  data  for  every  variable  it  simulates.  In  practice  this  is  not. 
realistic,  due  in  part  to  limits  on  time  and  money  for  developing  models  and 
limits  on  complexity  for  what  can  be  understood  and  put  to  work  by  the  user. 

8 • 2 Modeling  Teleoperato r Control , _es pecia lly  Supervisory  Control  of  Tel eopera tors . 

Probably  the  most  difficult  part  of  the  teleoperator  control  modeling  is  the 
manipulation  aspect.  Insofar  as  remote  vision,  communications  and  vehicle  control 
are  considered  part  of  teleoperation,  there  exist  corresponding  modeling  sub-disci- 
plines which  are  reasonably  well  developed  and  applied,  namely  signal  detection 
and  pattern  recognition,  information  theory,  and  conventional  control  theory, 
respectively.  This  is  not  true  of  manipulation. 
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One  might  think  of  manipulation  as  modelled  with  control  theory,  or  some 
mixture  of  same  with  classical  mechanics  and  the  theory  of  automata  (computers). 

To  affirm  this  is  to  affirm  hopes  and  aspirations,  for  our  theoretical  under- 
standing of  how  to  model  the  mechanics  and  control  of  grasping,  moving  and  assem- 
bling objects  is  primitive  indeed.  Manipulations  are  discontinous  in  time  and 
space.  One  can  stop  an  assembly  or  disassembly  task  in  the  middle  and  go  back 
to  it,  provided  enough  static  friction  exists.  Also,  there  are  logical  sequential 
contingencies  inherent  in  manipulation:  the  tool  must  be  located  before  it  can 
be  grasped,  grasped  and  positioned  before  it  can  be  used  on  another  object,  etc. 

Differential  equation  models  of  control  theory  don't  adapt  to  these  logical 
contingencies.  But  computer  programs  easily  adapt  to  such  contingencies  and 
can  simulate  continuous  Newtonian  mechanical  interactions.  The  problem  is  that 
such  simulation  models  tend  to  be  very  complex,  with  many  degrees  of  freedom  - 
"identification"  or  convergence  upon  parameters,  or  "solution"  in  a closed-form 
sense  is  very  difficult.  Modeling  the  behavior  of  a teleoperator  system  is  not  unlike 
modeling  the  motor  skills  of  a person;  the  inherent  difficulty  of  the  latter  is 
an  old  story  to  the  experimental  psychologist. 

When  an  active  computer  is  added  and  the  human  operator  becomes  a supervisory 
controller,  the  modelling  task  obviously  takes  on  new  dimensions  and  new  problems. 

It  is  appropriate  that  the  models  of  supervisory  control  strive  to  characterize 
(and  predict)  those  aspects  of  man-machine  behavior  which  are  unique  or  at  least 
different  (as  compared  to  teleoperation  in  general)  - such  as  what  tasks  the  com- 
puter can  do  best, and  what  performance  may  be  expected  from  human  operator  vs. 
computer  using  a common  measure,  when  the  human  operator  does  or  should  turn 
control  over  to  the  computer  and  vice  versa,  what  difficulties  are  experienced  by 
human  operator  and  computer  in  communicating  with  the  other. 

A key  question  which  models  might  help  answer  is  when  supervisory  control 
is  necessary  (or  better  than  non-supervisory  control,  or  economically  justifiable, 
etc.).  As  we  have  previously  suggested,  computer  automation  is  obviously  advanta- 
geous on  the  production  line  where  the  same  task  is  being  repeated  precisely.  A 
preprogrammed  device  can  move  faster  and  with  more  precision  when  the  environment 
is  known.  But  where  is  the  advantage  of  computer  control  in  undersea  tasks?  We 
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think  the  answer  lies  in  the  fact  that  some  elements  of  the  task  geometry,  etc., 
are  usually  known  ahead  of  time  arid  therefore  some  preprogrammed  elements  can  be 
called  up  usefully  in  almost  any  task.  Others,  of  course,  are  encountered 
afresh,  and  ingenuity  and  human  judgment  must  be  brought  to  bear  to  cope  with 
completely  unanticipated  events.  For  example,  if  holes  must  be  drilled  or  tapped, 
tools  must  be  exchanged,  cleaning  of  surfaces  or  scanning  with  instruments  must 
take  place  - these  operations  must  occur  mostly  in  a manner  which  can  be  antici- 
pated - except  for  locating  and  orienting  the  manipulator  (sensor)  relative 
to  the  environmental  object.  Thus,  once  this  manual  location  and  orientation  ac- 
tivity is  accomplished,  the  automatic  routines  can  be  called,  with  prospect  of  con- 
siderable savings  in  time  and  errors  relative  to  doing  the  whole  job  manually.  Mo- 
dels are  needed  to  generalize  on  experiments  to  help  decide  how  and  when  to  give 
control  to  human  vs.  computer. 

Figure  8.1  offers  a way  to  organize  our  discussion  of  models  currently  of 
promise  for  various  aspects  of  teleoperation,  expecially  supervisory  control  of 
teleoperation.  This  flow  chart  suggests  four  different  levels  at  which  decisions 
are  made,  each  including  a test  following  the  corresponding  decision  activity 
which  is  a basis  for  commencing  operations  at  the  next  lower  level.  At  the  lowest 
or  most  primitive  level  (A)  decisions  are  made  to  "sense  and  act"  quickly.  Such 
decisions  are  either  computer  program  controlled  or  they  are  perfunctory  rate  or 
position  servolng  by  the  human  operator.  At  this  level  feedback  is  essentially 
continuous.  At  the  next  higher  level  (B)  are  supervisory  control  decisions  and 
tests.  These  are  mostly  human,  though  sophisticated  supervisory  systems  may  in- 
clude computer  aids,  especially  in* testing  whether  programs  are  appropriate  before 
they  are  committed  to  actions.  Feedback  is  intermittent,  with  time  constants  of 
seconds  and  minutes.  At  the  highest  two  levels  ( C , D)  decisions  are  for  allocation 
and  design,  and  tests  are  almost  always  human.  Models,  as  suggested  earlier,  can 
be  used  to  describe  and  predict  events  at  each  of  these  four  levels.  Because  the 
events  at  A are  quite  different  from  those  at  B,  and  those  at  C and  D (taken 
together)  are  quite  different  in  turn,  we  have  chosen  to  separate  models  into 
these  three  categories. 
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8.3  Modeling  Teleoperator  System  Performance  at  the  Environment  Interface  (level  A) 


The  purpose  of  a teleoperator  is  to  manipulate  and/or  sense  objects  in  the 
environment.  The  key  variables  at  this  interface  are  the  space,  time  and  ^orce 
(and  energy)  components  of  this  continuous  time  interaction.  Thus  the  purpose  of 
models  of  this  interaction  is  to  predict  such  space,  time  and  force  events  - 
given  the  task  manipulation  and  control  configuration,  etc.  At  this  interface 
whether  a teleoperator  system  does  or  does  not  incorporate  supervisory  control 
can  be  a contingency  or  parametric  constant  inlhe  model,  but  it  need  not 
be  made  a variable. 

The  idea  of  the  Weber  fraction  for  sensing  or  motor  action  is  an  old  and 
still  viable  model:  the  probable  error  (of  distance,  time)  in  positioning 
movements  is  roughly  a constant  fraction  of  the  magnitude  of  the  corresponding 
variable,  down  to  relatively  small  movements. 

Breaking  motor  responses  into  gross  positioning  motions  followed  by  fine 
adjustment  motions  is  also  an  old  idea.  The  "scientific  management"  proponents 
of  the  1930's  (Taylor,  Gilbreth)  developed  a rather  elaborate  classification 
of  manipulations  (hand-object  interactions),  including  "grasp",  "transport 
loaded",  "transport  empty", 'position,  "preposition,  etc. 

Following  Shannon's  development  of  information  theory  in  the  late  forties, 
(Shannon  and  Weaver,  1963)  Fitts  (1954)  showed  how  the  logarithmic  measure  of  the 
ratio  of  move  distance  to  error  tolerance  made  a simple  but  useful  predictor  of 
the  move  time.  This  measure  (log  gr ror^'t o'l'e raf c e ^ a^s0  came  t0  l<nown  as  "Fitts1 
index  of  difficulty".  This  was  discussed  in  detail  in  Section  7. 

Ferrell  (1965)  showed  how  Fitts'  index  could  be  used  to  predict  how  many 
"open  loop"  moves  a human  subject  requires,  when  there  is  no  feedback,  to  move 
a certain  distance  to  within  a certain  tolerance.  He  went  on  to  show  how  the 
number  of  open  loop  moves  when  there  is  no  delay  can  be  used  predict  task  comple- 
tion time  when  there  is  a pure  transmission  time  delay  in  the  control  loop. 

A closely  related  application  of  information  theory  is  the  prediction  of 
response  time  as  a log  function  of  the  number  of  equiprobable  response  alterna- 
tives to  be  selected  among  - the  so  called  "disjunctive  reaction  time". 
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Combining  a stimulus  set  of  eventsof  differrng  probabilities  with  a set  of 
response  alternatives  with  differing  probabilities  and  required  movement  to 
within  a given  tolerance,  one  night  obtain  a crude  predictive  model. 

But  in  doing  assembly  in  multiple  degrees  of  freedom,  even  with  manipulating 
but  one  rigid  object  (say  a peg)  to  achieve  a final  state  relative  to  fixed 
environmental  contraints  (putting  it  in  a hole),  the  Fitts'  law  idea  can  easily 
break  down.  The  reason  is  that  reduction  of  uncertainty  does  not  normally  pro- 
ceed simultaneously  in  all  degrees  of  freedom.  Further,  there  are  special  non- 
linearities  encountered,  such  as  the  "binding"  phenomenon  investigated  by  Whitney 
(1978)  as  a function  of  angle  of  approach,  peg  and  hole  (and  tolerance)  dimen- 
sions, etc.  Whitney's  models  predict  "binding  or'jamrring"  situations  rather 
nicely  based  simply  on  kinematic  criteria. 

Classical  dynamic  models  of  manipulators  can  be  important  both  in  predicting 
oscillations  (which  tend  to  be  worst  with  arm  fully  extended  and  when  sudden  move- 
ments are  made)  and  as  a basis  for  determining  time-optimal  or  energy  optimal 
trajectories  to  move  the  arm  from  one  configuration  to  another.  Typically  the 
dynamic  limitations  of  the  manipulator  per  se  do  not  by  themselves  seem  to  limit 
performance;  in  undersea  situations  it  is  usually  the  operator's  ability  to  see 
and  control  precisely  which  sets  the  limit  on  accuracy  of  positioning  for  a given 
time,  or  of  time  required  to  position  to  a given  accuracy.  Sensory  threshold 
nonlinearies  added  with  a simple  control  loop  model  can  help  predict  such  per- 
formance limits. 

In  any  motor  skill  task,  and  this  is  necessarily  true  of  teleoperator  control 
a compromise  must  be  reached  between  time,  accuracy,  reliability  (errors)  and 
effort.  A simple  model  of  this  trade-off  is  a set  of  linear  constraining  re- 
lations (which  can  be  graphed  as  planes  in  hyperspace,  lines  if  only  two  trade- 
off variables  are  included  such  as  are  shown  in  Figure  8.2  ),  For  each  variable 
there  may  be  some  absolute  constraints  (i.e.,  no  matter  how  much  time  or  effort 
is  spent,  accuracy  can  be  no  better  than  some  hysteresis  constraint  or  visual 
error;  no  matter  what  accuracy  is  accepted  some  minimum  reaction  time  is  required) 
Given  such  a bounded  space  of  possible  solutions,  the  best  or  normative  linear  pro 
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FIGURE  8.2  SIMPLEX  LINEAR  PROGRAMMING  EXAMPLE  FOR  OPTIMIZING  A TELEOPERATOR, 
Given  a set  of  performance  tradeoff  curves  for  speed  and  accuracy, 
the  farther  to  upper  right  the  better.  Given  a set  of  absolute 
constraints  (e.g.,  maximum  slewing  speed  is  line  At, speed- 
accuracy  vibration  limit  is  line  A^),  then  X is  optimal  point, 
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gramming  model  is  that  point  having  the  maximum  value  on  a set  of  tradeoff 
curves  (i.e.,  the  objective  function).  This  is  called  a "simplex  graphical  so- 
lution". When  dynamical  constraining  equations  obtain, a dynamic  programming  ' 

solution  becomes  necessary. 

Thus  far  the  models  described  apply  equally  well  to  direct  and  supervisory 
control.  What  is  it  at  the  A level  interface  which  is  different  between  direct 
manual  and  supervisory  control  led  teleoperators?  One  difference  is  that  when  in 
the  supervisory  mode  significant  "dead  times"  appear  at  the  output  while  the 
operator  is  reprogramming.  Thus,  in  terms  of  the  completion  time  variable  alone, 
as  a function  of  some  "task  complexity"  attribute  (Figure  8.3),  at  some  degree 
of  complexity  supervisory  control  will  prove  superior  to  direct  manual  control. 

When  it  comes  to  accuracy  of  performance  direct  manual  control  may  be 
counted  on  to  have  some  minimum  probable  error,  but  never  be  grossly  in  error. 

The  supervisory  system,  on  the  other  hand,  can  easily  be  more  precise  when  it  has 
been  programmed  properly  and  environmental  contingencies  turn  out  t.o  be  as 
anticipated  or  have  been  allowed  for  in  the  program  (which  branches  and  adapts 
based  upon  the  teleoperator's  own  sensors).  But  occasionally  the  computer,  due 
to  its  own  failure  or  due  to  human  error  in  programming  it,  will  make  spectacular 
errors,  comparable  to  industrial  robots  which  proceed  with  apparant  precision 
to  assemble  parts  Which  are  never  picked  up,  or  to  spot  weld  the  thin  air. 

8.4  Modeling  Human  Operator  Behavior  at  the  Computer  Console  Interface  (level  B).  : 

] 

.} 

Unlike  models  at  level  A which  focus  on  the  ron^inous  manipulator/sensor-  \ 

to-environment  physical  interactions,  models  at  level  B focus  on  the  more  or 
less  discontinuous  communication  between  man  and  computer.  Such  communication, 
of  course,  is  not  present  in  direct  manual  teleoperator  control  in  a rate  or 
master-slave  mode.  For  the  latter  by  itself  the  A-B  level  differentiation  has 
little  point. 

A first  category  of  man-computer  interaction  to  be  modeled  is  the  use  of  the 
computer  in  planning ^ off-line  and  disconnected  from  the  teleoperator.  A use- 
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Figure  8.3  hypothetical  task  completion  times  as  a function  of  task  complexity. 

In  direct  teleoperator  control  completion  time  rises  smoothly  with 
task  complexity.  In  supervisory  control  an  initial  programming 
penalty  is  paid  on  each  move  no  matter  what  the  task  complexity, 
but  for  more  complex  tasks  there  is  a real  advantage. 


ful  model  should  help  compare  the  computer  with  other  (non  computer-based) 
planning  aids  used  in  supervisory  control,  including  no  planning  aids  at  all. 

There  can  be  several  meanings  of  "model"  here: 

1.  representations  of  how  an  analyst  structures  the  task  using  diagrams 
such  as  the  Sacerdoti  "procedural  net",  state  transition  diagrams,  logic 
flow 

2.  representations  of  how  the  operator  thinks  of  his  task  as  gleaned  from 
interviews  and  "verbal  protocol"  descriptions  (precendence  diagrams,  task 
time-lines,  goal  hierarchies) 

3.  how  the  operator  does,  or  might  use,  a computer-based  "internal  model" 
for  trial  and  error  thought  experiments,  or  as  a prediction  display  of 
future  events  extrapolated  from  present  conditions,  etc. 

4.  how  other  representations  or  memories  within  the  man-machine  system 
are  used  by  the  operator,  in  conjunction  with  his  own  "in-the-head"  inter- 
nal representation  of  the  task  end  the  current  state  of  the  system  (2  above); 
and  a computer-based  internal  mooe’  (3  above).  Such  additional  representa- 
tions can  be  embodied  in  the  current  configuration  of  a replica  controller 

or  the  current  status  of  a display.  Figure  8.4  points  out  the  variety  of 
such  "internal  models"  tne  operator  has  available. 

Probably  the  most  important  kind  of (external)  model  at  this  level  is  that  which 
characterizes  the  operator's  programming  and  control  decisions  - what  he  commands 
the  computer  to  do  and  what  he  controls  himself.  Such  models  would  seek  to 
predict: 

1)  what  part  of  task  the  operator  chooses  to  do  manually,  what  part  he 
programs  for  the  computer  to  do 

2)  what  commands  he  selects  from  among  those  available 

3)  in  what  order  or  with  what  contingencies  he  assembles  these 

4)  how  long  a string  of  commands  he  assembles  (how  far  operations  are  pro- 
grammed open  loop,  i.e.,  without  feedback).  There  is  an  analogy  to  sig- 
nal detection  theory  which  seems  to  apply  here  which  balances  the  mar- 
ginal progress  in  one  successful  program  against  the  increasing  risk 

of  tuiiure  as  the  program  becomes  longer. 

5)  what  balance  he  makes  between  sensing  and  motor  activities  in  specifying 
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FIGURE  8.4  LOCI  OF  "INTERNAL  MODELS"  IN  A TELEOPERATOR  SYSTEM.  Each  numbered 
location  in  one  way  or  another  represents  the  state  of  the  system, 
is  updated  either  by  the  operator  or  by  the  hardware/software, 
and  is  referenced  by  the  operator  in  planning  and  controlling. 
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commands.  The  "optimal  stopping"  modal  fits  here  - normatively  deter- 
mining a compromise  point  where  taking  more  data  to  better  understand 
the  problem  reaches  a point  of  diminishing  returns,  where  it  is  time  to 
start  action  before  it's  too  late. 

6)  how  long  it  takes  the  operator  to  write  such  a program 

7)  how  many  and  what  kinds  of  errors  he  makes  in  programming 

8}  what  procedure  the  operator  uses  for  discovering  and  debugging  an  inap- 
propriate string  of  commands,  i.e.  conmands  which  the  computer  rejects 

9)  the  extent  to  which  the  operator's  decisions  and  pace  are  determined 
by:  (a)  a nominal  plan  and  schedule;  (b)  conditioning  from  training 
or  by  other  experience;  (c)  ongoing  rational  decisions  about  what  to 
do  and  when 

After  the  operator  turns  his  program  over  to  the  computer  to  run  he  must 
monitor  its  execution.  Deciding  when  to  sample  the  input  and  when  to  reset  the 
controller  may  oe  modeled  in  terms  of  information  value  theory  (Sheridan,  1970) 
which  presupposes  an  internal  autocorrelation  model  of  input  events  plus  (possibly) 
a fading  memory  on  the  part  of  the  operator. 

Deciding  when  a particular  program  has  gone  awry  closely  resembles  the  pro- 
blem of  detecting  a change  in  process  dynamics  in  manual  control.  Thus  human 
recognition  performance  can  be  modeled  on  the  basis  of  on-line  process  identifi- 
cation (as  has  been  used  in  manual  control),  but  more  satisfactorily  (in  the 
teleoperator  context)  modeled  as  a statistical  deviation  between  observed  res- 
ponse and  that  of  an  internal  dynamic  model  (as  certain  failure  detection  models 
do) . 


An  interesting  experimental  question  is  whether  the  human  operator  can 
detect  a teleoperator  failure  better,  or  decide  on  a better  way  to  handle  a 
current  situation,  when  he  is  an  active  manual  controller  or  when  he  is  a 
passive  observer.  In  t.he  manual  control  (aircraft)  area  there  is  conflicting 
evidence  (Curry,  1976). 
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Another  type  of  man-computer  interaction  consists  in  the  operator  taking 
oyer  control  from  the  computer,  either  when  it  has  completed  its  task  or  in  an 
emergency.  This  can  be  done  by  a signal  to  the  computer  to  execute  a program 
which  interrupts  present  action  if  necessary,  and  returns  the  manipulator  to  a 
"safe"  take-over  configuration.  Alternatively  it  can  be  accomplished  by  in- 
stantaneously connecting  the  operator's  rate  or  master-slave  hand  control  to 
the  manipulator.  Modeling  this  recovery  situation  will  be  useful  in  comparing 
the  slower  and  more  orderly  (but  possibly  inadequate)  first  method  with  the  abrupt, 
erratic  second  method  which  nevertheless  may  be  more  apt  to  avoid  severe  error. 

Thus  models  at  this  level  explicitly  represent  the  trading  and  sharing  of 
control  between  human  operator  and  computer,  and  the  transitions  from  planning 
to  programming  to  monitoring  to  human  takeover.  Graphical  techniques  such  as 
were  described  in  the  chapter  on  task  analysis  - flow  charts,  transition  fre- 
quency charts,  etc.  - are  appropriate  here. 

8.5  Modeling  to  Decide  How  to  Allocate  Resources,  Design  Hardware  and  Software  (levels  C,__D) 

Models  at  this  level  step  back  still  farther  and  consider  systematically 
how  different  hardware  and  software  configurations  might  have  differential  effects 
on  operator  and  system  performance,  either  from  the  viewpoint  of  what  is  put  to- 
gether for  a given  mission,  or  what  is  designed  in  the  first  place. 

The  independent  variables  of  such  models  are  the  equipment  parameters  of 

1 . hardware 

a.  sensors  (range,  resolution,  etc.) 

b.  arms  (size,  accuracy,  speed,  power,  kinematics  and  anthropomorphism) 

c.  operator  console  and  display  - how  specialized,  wnat  form  of  control 

d.  maintainability 

2.  software 

a.  command  language  elements,  structure 

b.  speed  and  accuracy  of  sensing  and  control 

c.  alarms,  alarm  strategy 

d.  back  up  options 

e.  on-line  models  for  control 

f.  on-line  planning  aids 

g.  on-line  aids  for  training 
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The  "models",  then,  are  tables,  diagrams  or  analytic  or  computer-based 
methods  for  stating  what  effect  the  Inclusion  of  certain  hardware  or  software 
features,  or  more  or  less  automation,  or  better  or  worse  quality  of  any  such 
component,  might  have  on  system  performance  or  human  behavior  at  the  man-con- 
sole interface.  Models  at  this  level  would  explicitly  compare  different  mixes 
of  human  and  computer  decision  making. 

Tables  8.1  and  8,2  give  an  example,  A sequence  of  six  "decision  sub- 
elements" in  Table  8,1  is  assumed  to  apply  to  most  man-computer  decisions,  but 
there  is  a variety  of  ways  in  which  man  and  computer  can  cooperate.  Table  8.2 
orders  these  as  "levels  of  automation"  going  from  a level  wherein  the  human 
operator  does  everything  to  a level  where  the  computer  does  everything.  Clearly 
as  more  automation  is  introduced  some  benefits  accrue, but  concomitant  risks 
are  also  incurred.  The  model  can  help  the  designer  or  operational  manager 
decide  what  mix  of  man  and  computer  to  use. 

At  this  level  subjective  judgment  of  operators  or  observers  can  be  used 
to  advantage,  There  are  various  qualities  of  the  situation  which  can  be  modeled 
as  judgment  profiles,  such  as: 

1)  handling  qualities  (responsi veres s and  control abi 1 i ty  , interface  trans- 
parency, naturalness,  dexterity,  flexibility,  gracefulness  in  failure, 
etc.  as  suggested  inthe  previous  section). 

2)  operator  mental  workload 

Judgment  data  can  be  aggregated  and  scaled  on  single-dimensional  scales  or 
using  multi-dimensions.  There  are  different  "judgment  heuristic"  techniques  by 
which  such  subjectively  based  models  can  be  generated  (Sheridan,  1978).  These 
include: 

1)  simple  category  scales  of  a given  quality 

2)  utility  theory  (with  single  or  rnulti -attributed  arguments),  which 
forces  judges  to  scale  on  a ratio  basis,  such  as  "I'd  be  indifferent 
between  x for  sure  and  a 50  - 50  chance  of  y or  z" 

3)  policy  capturing,  which  assigns  overall  weights  directly  to  various 
points  in  mul ti -attribute  space  and  uses  linear  regression  to  specify 
the  relative  effects  of  various  levels  of  different  attributes 
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TABLE  8.1 

BEHAVIORAL  ELEMENTS  USED  TO  CHARACTERIZE 
DEGREES  OF  AUTOMATION  IN  MAN-COMPUTER  DECISION-MAKING 


OPERATORS  OPERANDS  OPERANDS 

COMBINE  WITH:  FOR  HUMAN  FOR  COMPUTER 

(control  coding)  (display  coding) 


REQUESTS 

(asks  from  other  party) 

options 

SELECT  action 

TELL  action 

GETS 

(fetches  what  is 
requested  or  necessary) 

options 

options 

SELECTS 

(chooses  from  among 
options  for  intended 
action) 

action 

action 

APPROVES 

(agrees  or  disagrees  with 
a particr  ar  decision) 

SELECT  action 

START  action 

START  action 

TELL  action 

STARTS 

(initiates  implementation) 

action 

action 

TELLS 

(informs  what  was  done) 

action 

l 

B 

B 

B 

E 

C 


B 

G 

B 

B 

II 

I! 


i 

i 


TABLE  8.2 

LEVELS  OF  AUTOMATION  IN  MAN-COMPUTER  DECISION-MAKING 


for  a single  elemental  decisive  step 


DESCRIPTION 

OF  INTERACTION 

HUMAN  COMPUTER 

FUNCTIONS  FUNCTIONS 

1 . human  does  the  whol e 
job  up  to  the  point  of  turn- 
ing it  over  to  the  computer 
to  implement. 

(GETS  options  from  outside) 

SELECTS  action 

\ 

STARTS  action 

2.  computer  helps  by  deter- 
mining the  options 

1 

(REQUESTS  options) 

GETS  options 

SELECTS  action 

♦ 

STARTS  action 

3.  computer  helps  determine 
options  and  suggests  one, 
which  human  need  not  follow. 

(REQUESTS  options) 

GETS  options 

(REQUESTS  SELECT  action) 

^ SELECTS  action 
SELECTS  action  (can  be"*"^ 
different) 

STARTS  action 

4.  computer  selects  action 
and  human  may  or  may  not 
do  it. 

J 

(REQUESTS  options) 

GETS  options 

/ 

(REQUESTS  SELECT  action) 

^SELECTS  action 

. x" 

APPROVES  SELECT  action  ^ 

1 

STARTS  action  if  HUMAN 

APPROVES 

1 


(REQUESTS  options) 


(REQUESTS  SELECT  action) 


APPROVES  START  action 


(REQUESTS  options) 


(REQUESTS  SELECT  action) 


APPROVES  START  action 


7.  computer  does  whole  job  (REQUESTS  SELECT  action) 
and  necessarily  tells  human 
what  it  did. 


8.  computer  does  whole  (REQUESTS  SELECT  action) 

job  and  tells  human  what  it 
did  only  if  human 
explicitly  asks. 


(REQUESTS  TELL  action) 


COMPUTER 


GETS  option 


SELECTS  action 


STARTS  action  if 
HUMAN  APPROVES 


GETS  options 


SELECTS  action 


STARTS  action  if 
HUMAN  APPROVES 
or  if  t > T and 
HUMAN  HAS  MOT  DISAPPROVED 


GETS  options 

t 

SELECTS  action 

I 

STARTS  action 
\ 

.TELLS  action 


GETS  options 

i 

SELECTS  action 

f 

STARTS  action 

I 

TELLS  action  if 
HUMAN  REQUESTS 


jj 

i 
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Thurstonian  scaling,  which  rescales  raw  data  along  a continuum  based 
on  discriminal  dispersion  (relative  spread)  of  the  judgments  for  a 
given  object  or  event 

5)  multi-dimensional  scaling,  which  utilizes  a matrix  of  "dissimilarity 
judgments"  between  objects  or  events  to  identify  the  principal  axes 
with  respect  to  which  dissimilarities  are  perceived 

6)  interpretive  structural  modeling,  a scheme  to  order  pairs  of  objects  or 
events  with  regard  to  some  diadic  relation  (e.g.,  "should  be  done  sooner 
than",  "affects  the  control  of")  which  presupposes  consistent  transi- 
tivity of  judgments  and  thereby  obviates  the  need  to  make  all  possible 
pair  comparisons. 

The  value  judgement:  from  individuals,  and  their  aggregation  into  "social 
choice"  models  of  group  values,  are  especially  important  in  dealing  with 
policy  questions,  such  as  when  should  the  computer  be  enabled  to  overrule 
the  operator.  No  general  answer  to  the  latter  question  is  available  for  now; 
there  are  examples  of  both  human  authority  over  computer  and  computer  authority- 
over  human  in  various  complex  and  high-risk  man-machine  systems. 
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9.  RESEARCH  NEEDS  FOR  MAN-COMPUTER  CONTROL  OF  UNDERSEA  TELEOPERATORS 

This  Section  summarizes  what  are  believed  to  be  primary  research  needs 
for  man-machine  control  of  undersea  teleoperators.  All  of  the  needs  cited 
have  been  implied  in  the  foregoing  sections;  they  are  presented  here  only 
in  capsule  form, 

9 . 1 Task  Analysis  and  Performance  Measurement 

In  view  of  the  close  relationship  between  task  analysis  and  performance 
measurement  of  teleoperators,  the  research  needs  in  these  two  areas  are  com- 
bined. Moreover,  because  tasks  and  the  tools  to  do  them  must  be  matched, 
research  is  necessary  to  further  clarify  this  task-teleoperator  matching 
relationship. 

1.  Continuing  efforts  are  needed  to  define  and  classify  undersea  tasks 

of  the  kinds  which  might  be  amenable  to  teleoperation,  (although  it  is  clear 
that  such  taxonomies  will  evolve  as  missions  evolve  and  as  teleoperator 
technology  changes  what  is  achievable  by  teleoperation).  Data  on  both 
sequential  contingencies  and  distributional  frequencies  should  be  compiled. 

Since  much  of  what  now  passes  for  task  analysis  is  compilation  of  anecdotal 
date,  there  is  a clear  need  to  observe,  measure  and  record  more  objectively 
and  precisely  what  is  now  done  or  attempted  by  divers  or  teleoperators, 

2,  From  an  operational  viewpoint  improved  methodology  for  analysis  of 
specific  undersea  missions  (e.g.,  search  a certain  ocean  area,  find  and 
retrieve  a particular  downed  aircraft)  and  specific  tasks  (e.g.,  secure  a 
net  around  fuselage)  is  important  in  order  to  decide: 

- what,  if  any,  teleoperator  system  to  employ,  or  if  a human  diver  is  better 

- what  "tool  kit"  should  accompany  the  teleoperator 

- what  kind  of  planning  and  preparation  to  do  and  what  are  the  support 

logistics 

- how  long  the  mission  is  likely  to  take 

- what  are  the  dollar  costs 

- what  are  the  risks  to  human  life,  failure  of  mission,  damage  to 

equipment,  etc. 
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3.  Improved  models  should  be  developed  for  the  physical  interactions 
between  teleoperator  sensors  and  manipulators  and  the  objects  they  sense 
and  manipulate.  Theories  of  pattern  recognition, signal  detection,  and 
others  are  being  applied  to  sensing.  There  is  little  or  no  suitable  theory 
of  manipulation,  including  both  mechanical  dynamics  and  control  logic, 
available. 

4.  In  cooperation  with  Navy  and  industrial  users,  vendor  companies  and 
the  research  conmunity,  an  accepted  battery  of  laboratory  tests  should  be 
developed  which  incorporates  a broad  range  of  features  of  "real"  undersea 
tasks.  These  tests  should  be  quantitatively  adjustable  or  calibratable 
with  respect  to  size,  force,  tolerance,  speed,  accuracy,  etc.,  required. 
They  should  quickly  yield  a profile  of  scores  on  salient  objective  perform- 
ance variables. 

5.  Subjective  measures  of  the  quality  of  various  phases  of  teleoperator 
control  (sensing,  command  programming.,  continuous  manual  control,  task 
execution  )should  also  be  developed. 

When  enough  test  data  are  accumulated  reliability  analyses  should  be 
performed,  including  both  human  operator  and  teleoperator  equipment  compon- 
ents . 

6.  Since  teleoperators  can  be  either  more  general-purpose  or  more  special- 
purpose,  research  is  needed  to  determine  when  the  "point  of  diminishing  gen 
erality"  is  reached,  assuming  generality  increases  cost  of  one  teleoperator 

9 . 2 Man-Computer  Communic ation 

1.  When  a computer  is  used  in  an  undersea  teleoperator  system  for  other 
than  real-time  aiding  of  sensing  or  control,  i.e.,  when  there  is  trading 
°f  control  between  human  operator  and  computer,  smooth  man-computer  comnuni 
cation  is  crucial.  The  sparse  evidence  available  suggests  that  when  this 
communication  is  awkward  supervisory  control  of  teleoperation  is  inferior 
to  direct  manual  control,  but  when  man-computer  communication  is  good 
supervisory  control  can  be  faster  and  more  precise.  This  advantage  of 
supervisory  control  is  especially  present  when  the  communication  channel 


to  the  teleoperator  is  degraded.  Not  only  can  research  in  this  broad  area 
be  of  great  advantage  to  improved  design  of  teleoperators,  but,  because  of 
the  generality  of  the  problem,  it  can  benefit  understanding  of  man-computer 
comnuni cation  in  general. 

2.  The  man  and  computer  must  understand  what  each  other  knows  and  is  intends 
ing  to  do.  Each  may  be  said  tp  have  an  "internal  model"  representing  the 
current  state  of  the  system  and  its  environment.  The  state  of  the  sensors 
and  displays  and  the  state  of  the  controls  are  also  available  models  or 
representations  of  current  knowledge.  Some  research  should  aim  to  under- 
stand how  man  and  computer  do  or  could  access  each  other  (plus  other  "models"), 
test  their  own  knowledge,  modify  their  own  or  update  the  other's  knowledge. 

3.  We  need  experiments  to  determine  how  people  structure  knowledge  about 
everyday  inspection  and  manipulation,  how  they  naturally  tell  other  people  how 
to  accomplish  such  tasks  or  describe  environmental  states,  what  metaphors, 
nouns,  verbs,  modifiers,  syntax  they  use.  We  need  experiments  to  determine 
how  people  perforin  when  their  means  for  communication  about  such  task  pro- 
cedures and  environmental  states  is  constrained  or  modified,  i.e.,  they  are 
restricted  in  their  symbolic  statements  or  analogic  commands.  Such  informa- 
tion can  then  be  used  to  devise  computer  knowledge  structures  which  best 
adapt  to  teleoperator  control  and  accord  with  human  ways  of  structuring 
knowledge  about  inspection  and  manipulation. 

4.  The  computer  may  perform  automatic  routines  to  search,  avoid  obstacles, 
accommodate  (make  fine  adjustments  to  fit  together  two  mating  parts  and  not 
bind ) .resolve  end-point  motions,  exchange  end-effector  tools,  move  in  a 
pattern  so  as  to  keep  a fixed  distance  from  a surface  or  a fixed  orientation 
relative  to  some  reference  frame, etc.  Beyond  "demonstration  of  special 
capabilities",  research  is  needed  to  show  when  these  capabilities  save  time  over 
direct  manual  control,  under  what  circumstances  the  operator  prefers  to  use 
them  at  the  (possible)  cost  of  extra  communication  burden,  and  what  the  risks 

of  failure  are, 

5.  "Trading"  and  "sharing"  as  discussed  earlier  in  this  report  are  very  differ- 
ent modes  of  working  with  another  human  being.  What  are  the  fundamental 
behavioral  traits  of  man  or  computer  which  militate  in  either  direction  for 
particular  types  of  task?  This  Is  a long  range  research  need. 

9-3 


6.  Research  should  be  done  to  understand  when  the  computer  activity  should 
be  transparent"  to  the  operator  (control  "sharing"  situations  where  0,'^y  the 
result  of  computer  processing  is  most  important)  and  when  the  computer 
activity  should  be  apparent  to  the  operator  (control  "trading"  situations  where 
computer  is  being  monitored  or  its  misbehavior  is  being  diagnosed). 

7.  Some  continuing  research  effort  should  be  devoted  to  better  means  to 
teach  sensors  how  to  search  or  manipulators  how  to  perform  using  combinations 
of  analogic  and  symbolic  commands.  Should  symbolic  commands  be  dedicated 
keys  or  be  strings  of  general  purpose  keys,  or  some  combination? 

8.  A persistent  research  question  concerns  when  the  human  operator 
should  have  authority  over  the  computer  and  when  the  computer  should  have 
author-jty  over  the  human  - and  on  what  time  sca^e. 

9.3  Sensing  and  Display 

Though  sensing  and  display  research  for  undersea  applications  is  active 
and  ongoing,  there  are  some  research  areas  pertaining  especially  to  man- 
computer  teleoperator  control. 

1,  One  research  opportunity  concerns  the  tradeoff  between  video  (or  sonar) 
frame  rate  and  resolution,  which  is  especially  critical  when  the  bandwidth  is 
low.  The  computer  can  allow  for  an  adjustable  tradeoff,  so  that  the  operator 
can  have  a more  or  less  continuous  but  low-resolution  picture  for  one  phase 
of  his  task  and  a very  occasional  high-resolution  picture  for  another  phase. 

2,  Limited  frame  rate,  when  combined  with  significant  transmission  time  delay 
(such  as  occurs  with  a sonic  communication  channel)  can  pose  severe  problems 

in  control.  One  solution  is  the  "predictor  display"  (discussed  in  Sections 
5 and  7).  Research  is  needed  on  performance  with  predictor  displays  for 
various  delays,  frame  rates,  process  dynamics  and  other  factors. 

3,  Computers  can  aid  t.eleoperator  displays  in  various  other  ways.  Super- 
posing sonic  and  video  images  might  be  an  advantage  in  turbid  water.  Super- 
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position  of  computer-generated  alphanumeric  or  graphic  information  over 
the  otherwise  conventional  video  display  can  preclude  unnecessary  eye 
scan  (much  as  an  aircraft  "head-up"  display  ).  Predictor  information 
could  be  superposed  on  the  video  picture,  as  could  sonar  range  information. 

4,  "Teleproprioception"  was  discussed  at  length  ir,  the  report  , and  we  feel 
it  warrants  considerable  further  research  beyond  the  sometimes  discouraging 
efforts  at  head- mounted  displays,  etc.  Computers  plus  storage  tubes  can 
provide  the  operator  a wide-angle  "local  model"  derived  from  previous  sweeps 
of  a narrow  angle  video  camera  or  a side-scan  sonar.  Replica  controllers 
can  serve  not  only  for  on-line  control  but  for  trial  runs  relative  to  "local 
three-dimensional  models".  Most  important  for  research  are:  (1)  a better 
theoretical  understanding  of  "teleproprioception",  and  (2)  a better  empirical 
data  base  to  specify  how  performance  degrades  as  correspondence  between 
remote  arm,  sensor  and  vehicle  and  local  counterparts  (arms  and  sensors  on 
both  operators'  body  and  video  display)  degrades. 

9.4  Continuous  Control 

There  are  a great  many  questions  regarding  continuous  teleoperator 
control  still  deserving  of  research  effort.  Among  these  are: 

1.  Usually  at  least  some  degrees -of -freedom  of  control  of  the  submersible 
vehicle  are  redundant  with  degrees -of- freedom  of  the  manipulator  arm.  With 
a s;x-degree-of-freedom  arm  and  six-degree-of-freedom  vehicle  control  there 
would  be  complete  redundancy.  Under  what  circumstances  should  redundancies 
be  eliminated  and  the  vehicle  propulsion  system  be  used  by  the  operator  to 
guide  the  arm? 

2.  Speech  recognition  and  soeech  production  are  now  technologically 
available.  Can  they  improve  human  control  of  a teleoperator? 

3.  A "replica  controller"  in  conjunction  with  a computer  can  be  used  as 
a position  controller  within  a given  envelope  and  a rate  controller  outside 
this  envelope  (see  description  of  the  M.I.T.  SUPERMAN  program  in  Section  6). 
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Some  decrees  of  freedom  of  the  replica  can  be  programmed  to  provide  position 
control  while  others  Drovide  rate  control.  Under  what  circumstances  are 
such  ‘'mixed  modes"  confusing  to  the  operator  and  what  are  their  advantages? 

4.  Force  reflection  is  now  available  on  some  undersea  manipulators,  but 
we  stiU  have  little  understanding  for  which  tasks  and  for  what  degrees  of 
freedom  force  reflection  is  important.  By  adding  force  reflection  to  some 
manipulator  degrees  of  freedom,  and  only  brakes  or  locks  to  the  other  degrees 
of  freedom,  can  the  same  effective  capability  as  with  full  force  reflection 
be  had  at.  lower  cost  and  complexity? 

5.  Manipulators  tend  to  have  constant  damping  for  a given  velocity, 
based  on  passive  damping  counterbalanced  by  rate  feed-forward.  They  could 
be  provided  adjustab.e  impedance  characteristics  to  allow,  for  example,  the 
operator  to  program  free  ballistic  (undamped)  motions  at  the  beginning  of 
large  excursions  and  heavily  damped  motion  at  the  end  or  for  fine  adjustments. 
Would  this  be  an  advantage? 

6.  Most  manual  control  research  has  been  done  with  linear  dynamic  processes. 
Teleoperators  present  classical  nonlinearities  about  which  there  is  still  a 
dearth  of  man-machine  dynamic  modeling  based  on  experiment.  Some  such  nonlinear- 
ities are  static  fr  ction,  backlash,  servo-bias,  gravity  droop  (arm  extended), 
slewing  rate  limits,  and  time  delay.  Sometimes  added  dynamic  constraints  may 
help  compensate  for  nonlinearities  (e.g,,  small  vibration  may  overcome  static 
friction,  viscoinertial  lag  may  prevent  time-delay  instability). 

7.  Finally,  research  is  needed  on  computer-control  strategies  for  "fail- 
soft"  abortions  in  case  there  is  evidence  that  the  control  loop  has  been 
opened  for  more  than  some  threshold  period,  or  some  human  input  is  obviously 
called-for  and  not  forthcoming.  These  may  continue  the  same  activity  at 

a lower  level,  may  force  retreat  to  a safety  position,  may  stop  and  "hold", 
or  stop  and  "relax",  or  may  begin  execution  of  a complex  return  and  recovery 
activity. 
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